Jul 10 00:58:32.221736 ip-10-0-128-136 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 00:58:32.221747 ip-10-0-128-136 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 00:58:32.221755 ip-10-0-128-136 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 00:58:32.222104 ip-10-0-128-136 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 00:58:42.427080 ip-10-0-128-136 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 00:58:42.427107 ip-10-0-128-136 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 745a1eff08dd473690e8a027e365a79f -- Jul 10 01:00:24.193333 ip-10-0-128-136 systemd[1]: Starting Kubernetes Kubelet... Jul 10 01:00:24.684294 ip-10-0-128-136 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 01:00:24.684294 ip-10-0-128-136 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 01:00:24.684294 ip-10-0-128-136 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 01:00:24.684294 ip-10-0-128-136 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 01:00:24.684294 ip-10-0-128-136 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 01:00:24.685414 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.685312 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 01:00:24.690524 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690502 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.690524 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690523 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690527 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690531 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690536 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690539 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690542 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690545 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690548 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690551 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690554 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690557 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690561 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690564 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690566 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690569 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690572 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690575 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690577 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690580 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690588 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.690606 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690591 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690593 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690598 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690603 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690606 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690609 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690613 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690616 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690620 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690623 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690626 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690629 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690631 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690634 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690638 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690641 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690644 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690646 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690649 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.691145 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690652 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690656 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690660 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690663 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690666 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690668 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690671 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690674 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690677 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690681 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690684 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690686 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690689 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690692 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690695 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690699 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690702 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690705 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690708 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.691644 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690710 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690713 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690716 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690718 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690721 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690724 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690729 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690733 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690735 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690738 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690740 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690743 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690746 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690748 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690751 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690753 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690756 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690759 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690761 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690764 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.692165 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690766 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690769 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690772 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690775 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690777 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690781 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690783 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690788 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690792 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690795 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690798 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690801 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690804 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690806 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690809 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690812 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690815 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690817 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690820 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.690823 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.692686 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692668 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692678 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692683 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692688 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692692 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692695 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692699 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692702 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692705 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692708 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692711 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692714 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692717 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692720 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692722 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692725 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692728 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692731 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692734 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692737 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.693235 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692740 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692745 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692749 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692752 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692755 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692759 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692762 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692765 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692768 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692770 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692775 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692778 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692781 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692784 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692788 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692792 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692795 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692798 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692801 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.693812 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692804 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692807 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692810 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692813 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692816 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692818 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692821 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692825 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692827 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692830 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692833 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692836 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692839 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692841 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692845 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692848 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692850 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692853 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692856 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.694338 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692858 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692861 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692864 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692866 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692869 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692872 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692874 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692878 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692882 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692885 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692888 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692890 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692893 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692896 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692898 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692901 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692903 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692906 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692909 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692911 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.694841 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692914 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692917 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692919 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692922 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692925 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692928 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692930 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692933 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692940 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692943 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692946 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692949 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692951 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692954 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692957 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692960 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692979 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692982 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692985 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692988 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.695395 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.692991 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693403 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693409 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693413 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693416 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693420 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693423 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693426 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693430 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693434 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693438 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693441 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693443 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693446 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693449 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693452 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693458 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693460 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693464 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693467 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.695915 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693474 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693477 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693480 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693483 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693485 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693488 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693491 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693493 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693496 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693499 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693501 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693504 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693511 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693514 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693516 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693519 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693522 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693524 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693527 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.696465 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693530 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693532 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693535 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693538 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693540 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693543 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693546 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693549 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693552 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693555 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693558 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693561 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693564 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693566 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693570 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693572 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693575 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693577 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693580 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693583 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.696997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693585 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693588 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693590 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693593 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693596 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693600 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693603 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693606 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693609 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693612 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693614 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693617 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693620 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693622 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693625 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693628 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693630 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693633 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693635 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693638 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.697531 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693641 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693645 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693647 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693650 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693653 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693655 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693658 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693661 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693663 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693666 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693669 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693671 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693674 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693676 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693679 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693681 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693684 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693689 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693692 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693695 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.698069 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.693697 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693775 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693783 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693791 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693796 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693800 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693803 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693808 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693813 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693817 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693820 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693824 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693828 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693832 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693835 2565 flags.go:64] FLAG: --cgroup-root="" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693838 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693841 2565 flags.go:64] FLAG: --client-ca-file="" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693845 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693848 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693852 2565 flags.go:64] FLAG: --cluster-domain="" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693857 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693860 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693863 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693867 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 01:00:24.698632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693871 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693874 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693877 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693881 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693884 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693888 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693892 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693895 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693898 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693902 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693905 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693908 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693911 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693915 2565 flags.go:64] FLAG: --enable-server="true" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693918 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693923 2565 flags.go:64] FLAG: --event-burst="100" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693926 2565 flags.go:64] FLAG: --event-qps="50" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693929 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693933 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693936 2565 flags.go:64] FLAG: --eviction-hard="" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693940 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693944 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693947 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693951 2565 flags.go:64] FLAG: --eviction-soft="" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693954 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 01:00:24.699282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693957 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693960 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693981 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693985 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693989 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693992 2565 flags.go:64] FLAG: --feature-gates="" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.693996 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694000 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694003 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694006 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694010 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694013 2565 flags.go:64] FLAG: --help="false" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694018 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-128-136.ec2.internal" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694021 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694024 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694027 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694031 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694034 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694037 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694040 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694044 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694047 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694050 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694054 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 01:00:24.699957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694057 2565 flags.go:64] FLAG: --kube-reserved="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694060 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694063 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694066 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694069 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694072 2565 flags.go:64] FLAG: --lock-file="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694076 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694079 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694082 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694087 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694090 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694094 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694097 2565 flags.go:64] FLAG: --logging-format="text" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694100 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694103 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694106 2565 flags.go:64] FLAG: --manifest-url="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694109 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694114 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694117 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694121 2565 flags.go:64] FLAG: --max-pods="110" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694126 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694129 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694132 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694135 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694138 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 01:00:24.700615 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694141 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694144 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694153 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694156 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694159 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694162 2565 flags.go:64] FLAG: --pod-cidr="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694165 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694171 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694174 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694177 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694180 2565 flags.go:64] FLAG: --port="10250" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694184 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694187 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e938bc4e3df2bf9" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694190 2565 flags.go:64] FLAG: --qos-reserved="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694194 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694197 2565 flags.go:64] FLAG: --register-node="true" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694200 2565 flags.go:64] FLAG: --register-with-taints="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694203 2565 flags.go:64] FLAG: --registry-burst="10" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694206 2565 flags.go:64] FLAG: --registry-qps="5" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694210 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694212 2565 flags.go:64] FLAG: --reserved-memory="" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694217 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694220 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694223 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694227 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 01:00:24.701300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694230 2565 flags.go:64] FLAG: --runonce="false" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694233 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694236 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694241 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694244 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694246 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694250 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694253 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694256 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694259 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694262 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694265 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694268 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694271 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694275 2565 flags.go:64] FLAG: --system-cgroups="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694278 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694283 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694286 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694291 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694295 2565 flags.go:64] FLAG: --tls-min-version="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694298 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694301 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694304 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694307 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694310 2565 flags.go:64] FLAG: --v="2" Jul 10 01:00:24.701988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694315 2565 flags.go:64] FLAG: --version="false" Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694319 2565 flags.go:64] FLAG: --vmodule="" Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694323 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.694327 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694454 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694458 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694461 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694465 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694469 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694473 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694477 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694480 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694482 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694485 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694488 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694491 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694494 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694497 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694500 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694503 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.702775 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694505 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694510 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694514 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694517 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694520 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694524 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694527 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694530 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694533 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694536 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694539 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694541 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694544 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694546 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694549 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694552 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694554 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694558 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694560 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.703641 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694563 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694566 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694568 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694572 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694575 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694578 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694581 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694584 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694586 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694589 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694592 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694595 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694598 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694601 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694603 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694606 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694609 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694613 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694617 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694620 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.704275 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694622 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694625 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694628 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694630 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694633 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694636 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694638 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694641 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694644 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694646 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694649 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694652 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694655 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694658 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694661 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694666 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694670 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694673 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694676 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694679 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.704804 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694681 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694685 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694687 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694690 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694693 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694695 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694698 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694701 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694704 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694706 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694711 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694713 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694716 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694719 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694722 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694724 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694727 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694730 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694732 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694735 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.705467 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694738 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694741 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694743 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.694746 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.695605 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.705132 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.705154 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.705178 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.705184 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705252 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705258 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705262 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705275 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705281 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.706006 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705286 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705289 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705293 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705296 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705300 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705303 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705307 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705310 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705313 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705317 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705320 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705323 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705326 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705329 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705332 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705335 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705338 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705341 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705344 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705347 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.706393 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705350 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705352 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705355 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705359 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705363 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705366 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705369 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705372 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705374 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705377 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705379 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705382 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705385 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705388 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705391 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705393 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705396 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705398 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705401 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.706946 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705404 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705406 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705409 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705412 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705415 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705417 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705420 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705422 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705425 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705428 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705432 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705434 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705437 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705439 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705442 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705445 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705447 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705450 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705453 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705456 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.707471 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705458 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705461 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705465 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705469 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705472 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705475 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705477 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705480 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705484 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705487 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705489 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705492 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705495 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705497 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705500 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705503 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705506 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705509 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705512 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705514 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.708034 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705517 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705521 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705524 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705527 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705529 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705532 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705535 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705539 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705543 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705546 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705549 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705553 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705556 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705558 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.708574 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705561 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.705567 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705698 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705703 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705707 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705710 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705713 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705716 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705719 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705722 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705725 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705727 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705730 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705734 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 01:00:24.708947 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705739 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705742 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705745 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705748 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705752 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705755 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705759 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705761 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705764 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705767 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705770 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705773 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705775 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705778 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705781 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705784 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705787 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705789 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705792 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 01:00:24.709449 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705795 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705797 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705800 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705802 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705806 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705809 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705811 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705814 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705817 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705819 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705824 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705827 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705830 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705833 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705836 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705839 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705842 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705845 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705848 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705850 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 01:00:24.709982 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705853 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705856 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705859 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705861 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705864 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705867 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705869 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705872 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705875 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705878 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705881 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705883 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705886 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705888 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705891 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705894 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705897 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705899 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705902 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705905 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 01:00:24.710508 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705907 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705910 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705912 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705915 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705918 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705920 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705923 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705927 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705929 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705932 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705935 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705937 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705940 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705942 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705945 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705947 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705953 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705955 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705958 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 01:00:24.711053 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705961 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705979 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705983 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705986 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705989 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705991 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705997 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:24.705999 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.706005 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.706158 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.708814 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.710989 2565 server.go:1019] "Starting client certificate rotation" Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.711105 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 01:00:24.711575 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.711146 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 01:00:24.713918 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.713901 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 01:00:24.715304 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.715285 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 01:00:24.728668 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.728640 2565 log.go:25] "Validated CRI v1 runtime API" Jul 10 01:00:24.733163 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.733136 2565 log.go:25] "Validated CRI v1 image API" Jul 10 01:00:24.734779 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.734761 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 01:00:24.740338 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.740309 2565 fs.go:135] Filesystem UUIDs: map[78a74aeb-9564-4e73-9442-4d1801a16b69:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 895f85f7-31aa-4995-bc56-159838934432:/dev/nvme0n1p3] Jul 10 01:00:24.740448 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.740348 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 01:00:24.743722 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.743696 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 01:00:24.748763 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.747919 2565 manager.go:216] Machine: {Timestamp:2026-07-10 01:00:24.745985829 +0000 UTC m=+0.407780168 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3293289 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec214f87ac77770421363930d66d9210 SystemUUID:ec214f87-ac77-7704-2136-3930d66d9210 BootID:745a1eff-08dd-4736-90e8-a027e365a79f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b7:3a:20:44:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b7:3a:20:44:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:1b:42:8b:d9:b5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 01:00:24.748763 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.748377 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 01:00:24.748763 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.748525 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 01:00:24.752209 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.752172 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 01:00:24.752406 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.752214 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-136.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 01:00:24.753167 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.753153 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 01:00:24.753225 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.753171 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 01:00:24.753225 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.753184 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 01:00:24.753225 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.753207 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 01:00:24.755083 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.755066 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 01:00:24.760035 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760016 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 10 01:00:24.760201 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760187 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 01:00:24.760277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760236 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 10 01:00:24.760277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760253 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 01:00:24.760370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760281 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 01:00:24.760370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760297 2565 kubelet.go:398] "Adding apiserver pod source" Jul 10 01:00:24.760370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760313 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 01:00:24.760786 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760771 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 01:00:24.760863 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.760796 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 01:00:24.765753 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.765730 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 01:00:24.767663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.767641 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cqz2d" Jul 10 01:00:24.768375 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768359 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 01:00:24.768436 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768394 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 01:00:24.768612 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768600 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 01:00:24.768648 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768618 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 01:00:24.768648 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768624 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 01:00:24.768648 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768631 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 01:00:24.768648 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768639 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 01:00:24.768648 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.768645 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 01:00:24.769219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769208 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 01:00:24.769219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769219 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 01:00:24.769288 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769226 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 01:00:24.769288 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769232 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 01:00:24.769288 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769241 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 01:00:24.769288 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.769249 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 01:00:24.770127 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.770114 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 01:00:24.770127 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.770125 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 01:00:24.771493 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.771468 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-136.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 01:00:24.771620 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.771598 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 01:00:24.773924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.773910 2565 server.go:1268] "Started kubelet" Jul 10 01:00:24.774029 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.773996 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 01:00:24.774093 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.774033 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 01:00:24.774183 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.774155 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 01:00:24.774950 ip-10-0-128-136 systemd[1]: Started Kubernetes Kubelet. Jul 10 01:00:24.775356 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.775328 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cqz2d" Jul 10 01:00:24.775798 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.775781 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 01:00:24.775869 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.775856 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 10 01:00:24.781809 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.781775 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 01:00:24.782389 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.782367 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-136.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 01:00:24.782719 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.782701 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 01:00:24.782854 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.782787 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 01:00:24.782926 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.782814 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:24.783531 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783507 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 01:00:24.783635 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.782456 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-136.ec2.internal.18c0c7e1bbf7dbca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-136.ec2.internal,UID:ip-10-0-128-136.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-136.ec2.internal,},FirstTimestamp:2026-07-10 01:00:24.77386849 +0000 UTC m=+0.435662838,LastTimestamp:2026-07-10 01:00:24.77386849 +0000 UTC m=+0.435662838,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-136.ec2.internal,}" Jul 10 01:00:24.783837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783814 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 01:00:24.783837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783838 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 01:00:24.783978 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783825 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 01:00:24.783978 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783920 2565 factory.go:55] Registering systemd factory Jul 10 01:00:24.783978 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.783932 2565 factory.go:223] Registration of the systemd container factory successfully Jul 10 01:00:24.784098 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784017 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 10 01:00:24.784098 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784027 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 10 01:00:24.784206 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784193 2565 factory.go:153] Registering CRI-O factory Jul 10 01:00:24.784242 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784211 2565 factory.go:223] Registration of the crio container factory successfully Jul 10 01:00:24.784242 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784238 2565 factory.go:103] Registering Raw factory Jul 10 01:00:24.784328 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784251 2565 manager.go:1196] Started watching for new ooms in manager Jul 10 01:00:24.784721 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.784691 2565 manager.go:349] Starting recovery of all containers Jul 10 01:00:24.788731 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.788702 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:24.792175 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.791910 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-136.ec2.internal\" not found" node="ip-10-0-128-136.ec2.internal" Jul 10 01:00:24.798502 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.798477 2565 manager.go:354] Recovery completed Jul 10 01:00:24.803515 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.803494 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:24.807130 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807107 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:24.807212 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807146 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:24.807212 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807160 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:24.807776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807755 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 01:00:24.807776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807772 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 01:00:24.807949 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807793 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 10 01:00:24.807949 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.807794 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:24.808722 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.808702 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:24.808780 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.808740 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:24.808780 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.808757 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:24.809635 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.809616 2565 policy_none.go:49] "None policy: Start" Jul 10 01:00:24.809635 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.809636 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 01:00:24.809748 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.809647 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 01:00:24.811293 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.811280 2565 policy_none.go:47] "Start" Jul 10 01:00:24.850681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.850660 2565 manager.go:337] "Starting Device Plugin manager" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.850710 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.850722 2565 server.go:87] "Starting device plugin registration server" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.851017 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.851029 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.851139 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.851239 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.851248 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.851827 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 01:00:24.872028 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.851873 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:24.935317 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.935223 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 01:00:24.936776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.936754 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 01:00:24.936856 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.936809 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 01:00:24.936856 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.936843 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 01:00:24.936954 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.936891 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 01:00:24.939737 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.939712 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:24.952277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.952233 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:24.953537 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.953508 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:24.953675 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.953547 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:24.953675 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.953563 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:24.953675 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.953591 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-136.ec2.internal" Jul 10 01:00:24.965424 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:24.965391 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-136.ec2.internal" Jul 10 01:00:24.965424 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.965424 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-136.ec2.internal\": node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:24.980049 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:24.980017 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.040357 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.040290 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal"] Jul 10 01:00:25.040456 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.040416 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:25.042518 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.042496 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:25.042638 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.042536 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:25.042638 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.042553 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:25.044984 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.044962 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:25.045093 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045055 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.045093 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045085 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:25.045876 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045857 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:25.045975 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045880 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:25.045975 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045925 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:25.045975 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045938 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:25.045975 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045895 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:25.045975 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.045968 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:25.048232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.048214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.048304 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.048248 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 01:00:25.049031 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.049014 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientMemory" Jul 10 01:00:25.049128 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.049047 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 01:00:25.049128 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.049059 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeHasSufficientPID" Jul 10 01:00:25.071987 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.071957 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-136.ec2.internal\" not found" node="ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.077151 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.077131 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-136.ec2.internal\" not found" node="ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.080264 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.080243 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.180344 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.180305 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.185370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.185306 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78350d0908fbb8795cb0dc5d10a96325-config\") pod \"kube-apiserver-proxy-ip-10-0-128-136.ec2.internal\" (UID: \"78350d0908fbb8795cb0dc5d10a96325\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.185370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.185342 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.185370 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.185364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.281244 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.281197 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.286002 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.285971 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78350d0908fbb8795cb0dc5d10a96325-config\") pod \"kube-apiserver-proxy-ip-10-0-128-136.ec2.internal\" (UID: \"78350d0908fbb8795cb0dc5d10a96325\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.286110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.286007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78350d0908fbb8795cb0dc5d10a96325-config\") pod \"kube-apiserver-proxy-ip-10-0-128-136.ec2.internal\" (UID: \"78350d0908fbb8795cb0dc5d10a96325\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.286110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.286049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.286110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.286066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.286110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.286094 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.286252 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.286140 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e6349be74bf44ff9b94e695d15114e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal\" (UID: \"3e6349be74bf44ff9b94e695d15114e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.373817 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.373768 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.380185 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.380152 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.382346 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.382323 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.486856 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.486760 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.587942 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.587909 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.636315 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.636284 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:25.688593 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.688559 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.709551 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.709500 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 01:00:25.709742 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.709717 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 01:00:25.709821 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.709742 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 01:00:25.709821 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.709742 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 01:00:25.783652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.783533 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 00:55:25 +0000 UTC" deadline="2028-01-01 08:02:57.669432097 +0000 UTC" Jul 10 01:00:25.783652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.783595 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12967h2m31.885842055s" Jul 10 01:00:25.789671 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:25.789648 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-136.ec2.internal\" not found" Jul 10 01:00:25.854947 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.854916 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:25.883938 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.883879 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.895214 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.894718 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 01:00:25.896225 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.896204 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" Jul 10 01:00:25.903048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:25.903028 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 01:00:26.018217 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:26.018183 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78350d0908fbb8795cb0dc5d10a96325.slice/crio-3044aacaac7d3f94199bb6bfd735fde0c4e49b12e712c7129c360565512986c1 WatchSource:0}: Error finding container 3044aacaac7d3f94199bb6bfd735fde0c4e49b12e712c7129c360565512986c1: Status 404 returned error can't find the container with id 3044aacaac7d3f94199bb6bfd735fde0c4e49b12e712c7129c360565512986c1 Jul 10 01:00:26.023811 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.023789 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 01:00:26.028628 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:26.028601 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6349be74bf44ff9b94e695d15114e3.slice/crio-a23a6de4dc66811d3cf241bd8354a0f4ddbb379033a900ad736a7fbb1c14e225 WatchSource:0}: Error finding container a23a6de4dc66811d3cf241bd8354a0f4ddbb379033a900ad736a7fbb1c14e225: Status 404 returned error can't find the container with id a23a6de4dc66811d3cf241bd8354a0f4ddbb379033a900ad736a7fbb1c14e225 Jul 10 01:00:26.760946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.760906 2565 apiserver.go:52] "Watching apiserver" Jul 10 01:00:26.768570 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.768522 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 01:00:26.769172 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.769140 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal","openshift-multus/multus-additional-cni-plugins-k8frj","openshift-multus/multus-f9kxd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs","openshift-image-registry/node-ca-5kcqk","openshift-multus/network-metrics-daemon-rztvp","openshift-network-diagnostics/network-check-target-k2xjk","openshift-network-operator/iptables-alerter-brfdr","openshift-ovn-kubernetes/ovnkube-node-cqgp9","kube-system/global-pull-secret-syncer-x5cp6","kube-system/konnectivity-agent-l7v52","kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hdwcf","openshift-dns/node-resolver-q9pks"] Jul 10 01:00:26.772276 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.771968 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:26.772276 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.772072 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:26.774555 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.774529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.778412 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.776842 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.779939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.778899 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pjnd5\"" Jul 10 01:00:26.779939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.779025 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.779939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.779163 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.779939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.779484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.781159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.780509 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.781159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.780551 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 01:00:26.781159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.780735 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4q2mf\"" Jul 10 01:00:26.781159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.780766 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.781159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.780960 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 01:00:26.781476 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.781415 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.782123 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.782101 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.782379 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.782360 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 01:00:26.782573 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.782509 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.782650 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.782579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zgw7f\"" Jul 10 01:00:26.783800 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.783727 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 01:00:26.783899 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.783865 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.784007 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.783967 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:26.786439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.785074 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 01:00:26.786439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.785354 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.786439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.785644 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.786439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.785832 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4bgrl\"" Jul 10 01:00:26.786837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.786768 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.789410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.789499 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2mp9k\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.789659 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.789824 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.790020 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.790060 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.790074 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.790102 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.792535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.791949 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.798787 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-57mcq\"" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.801179 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.801537 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.801797 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.802164 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802518 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-768ft\"" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-socket-dir-parent\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-multus-daemon-config\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802627 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-node-log\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-netd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-conf\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-os-release\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-cni-binary-copy\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802783 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-etc-tuned\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802815 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w9kr\" (UniqueName: \"kubernetes.io/projected/5227952a-01f8-411a-870b-b208ca062c27-kube-api-access-2w9kr\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-device-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894b8144-59f5-4387-b59b-4564ea772cf7-host\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.803289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802940 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.802991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803020 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-modprobe-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-bin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803098 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803146 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-sys\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803214 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-var-lib-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803420 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-env-overrides\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-systemd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-systemd\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f54p\" (UniqueName: \"kubernetes.io/projected/bafc5e2d-4876-4e8a-9c9d-740a854045c2-kube-api-access-5f54p\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803560 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-ovn\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803593 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-k8s-cni-cncf-io\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804307 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-kubelet\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803750 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-multus\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803783 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovn-node-metrics-cert\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntmvc\" (UniqueName: \"kubernetes.io/projected/894b8144-59f5-4387-b59b-4564ea772cf7-kube-api-access-ntmvc\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-netns\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-log-socket\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.803989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-config\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-run\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804070 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-cnibin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804100 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-etc-kubernetes\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h969f\" (UniqueName: \"kubernetes.io/projected/cf888257-f4f1-47c2-a54b-eba014c092b6-kube-api-access-h969f\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-systemd-units\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-bin\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-tmp\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-var-lib-kubelet\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.804865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-etc-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-multus-certs\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804491 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804583 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysconfig\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804648 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-lib-modules\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-netns\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804702 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804724 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpb4j\" (UniqueName: \"kubernetes.io/projected/ce04d9bb-c7f1-48ee-879a-32cac449f751-kube-api-access-lpb4j\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-system-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804846 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-kubelet\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-hostroot\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-conf-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.804958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894b8144-59f5-4387-b59b-4564ea772cf7-serviceca\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-slash\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-script-lib\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.805663 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-kubernetes\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.806606 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-host\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.806606 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805109 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 01:00:26.806606 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvpx5\" (UniqueName: \"kubernetes.io/projected/218c03bf-de10-4fbe-88ba-91869f85e495-kube-api-access-jvpx5\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.806606 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.805433 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:26.809603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.807690 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 01:00:26.809603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.808364 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 01:00:26.809603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.808584 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bvvhr\"" Jul 10 01:00:26.809603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.808924 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:26.811447 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.811424 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 01:00:26.813358 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.812469 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 01:00:26.813358 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.812546 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 01:00:26.813358 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.812805 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qwqqc\"" Jul 10 01:00:26.819017 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.818989 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p7pwv" Jul 10 01:00:26.824782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.824752 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p7pwv" Jul 10 01:00:26.885412 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.885379 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 01:00:26.905508 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntmvc\" (UniqueName: \"kubernetes.io/projected/894b8144-59f5-4387-b59b-4564ea772cf7-kube-api-access-ntmvc\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-netns\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905546 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-log-socket\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905573 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-config\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-run\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905621 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-cnibin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-etc-kubernetes\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.905686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h969f\" (UniqueName: \"kubernetes.io/projected/cf888257-f4f1-47c2-a54b-eba014c092b6-kube-api-access-h969f\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-systemd-units\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905716 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-bin\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91de7b74-be93-4945-9d06-c036c40e4f1b-agent-certs\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1833f960-875b-4907-90c8-bcdb3c5ffb41-hosts-file\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-tmp\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905839 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91de7b74-be93-4945-9d06-c036c40e4f1b-konnectivity-ca\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3405dad4-9e04-4674-a806-2fd4e09b4966-host-slash\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-system-cni-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.905988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.905983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-var-lib-kubelet\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-etc-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-kubelet-config\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cnibin\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbmxb\" (UniqueName: \"kubernetes.io/projected/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-kube-api-access-pbmxb\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906103 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-multus-certs\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysconfig\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-lib-modules\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906237 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-netns\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpb4j\" (UniqueName: \"kubernetes.io/projected/ce04d9bb-c7f1-48ee-879a-32cac449f751-kube-api-access-lpb4j\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.906349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-system-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-kubelet\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-hostroot\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-conf-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894b8144-59f5-4387-b59b-4564ea772cf7-serviceca\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-slash\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906488 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-script-lib\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-kubernetes\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-host\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvpx5\" (UniqueName: \"kubernetes.io/projected/218c03bf-de10-4fbe-88ba-91869f85e495-kube-api-access-jvpx5\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-socket-dir-parent\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-multus-daemon-config\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-node-log\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-netd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7247g\" (UniqueName: \"kubernetes.io/projected/1833f960-875b-4907-90c8-bcdb3c5ffb41-kube-api-access-7247g\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906741 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-conf\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-os-release\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-cni-binary-copy\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.906837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmkh\" (UniqueName: \"kubernetes.io/projected/3405dad4-9e04-4674-a806-2fd4e09b4966-kube-api-access-4bmkh\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-etc-tuned\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w9kr\" (UniqueName: \"kubernetes.io/projected/5227952a-01f8-411a-870b-b208ca062c27-kube-api-access-2w9kr\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3405dad4-9e04-4674-a806-2fd4e09b4966-iptables-alerter-script\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.906983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-device-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894b8144-59f5-4387-b59b-4564ea772cf7-host\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-modprobe-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907127 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-bin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907183 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1833f960-875b-4907-90c8-bcdb3c5ffb41-tmp-dir\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-sys\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907239 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.907681 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907293 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-var-lib-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-env-overrides\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907387 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907410 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907430 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-systemd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-systemd\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907485 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f54p\" (UniqueName: \"kubernetes.io/projected/bafc5e2d-4876-4e8a-9c9d-740a854045c2-kube-api-access-5f54p\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907567 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-ovn\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-k8s-cni-cncf-io\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-kubelet\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-dbus\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907676 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-os-release\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:26.908409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-multus\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.907722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovn-node-metrics-cert\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908170 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908385 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-socket-dir-parent\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-netns\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908750 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-log-socket\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908815 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908872 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894b8144-59f5-4387-b59b-4564ea772cf7-host\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908925 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.908992 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909117 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-modprobe-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909167 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-bin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909220 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-multus-daemon-config\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.909261 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909290 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-config\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909306 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-node-log\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.909343 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:27.409306052 +0000 UTC m=+3.071100405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909355 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-netd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909355 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-device-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909390 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-run\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909425 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-cnibin\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909460 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-etc-kubernetes\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909501 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-conf\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909602 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-os-release\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909670 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-systemd-units\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.909693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.909702 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-cni-bin\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.910553 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910224 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5227952a-01f8-411a-870b-b208ca062c27-cni-binary-copy\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.910553 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-lib-modules\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.910553 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910435 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.910553 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910519 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-var-lib-kubelet\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.910736 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910567 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-etc-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.910736 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910655 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-multus-certs\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.910988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.910948 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysconfig\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.911125 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911083 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-sysctl-d\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.911271 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911253 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-multus-conf-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.911330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-netns\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.911458 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911423 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.911735 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911716 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-system-cni-dir\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.911798 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911775 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-kubelet\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.911862 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.911837 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-hostroot\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912395 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-sys\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912473 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912568 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912620 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-var-lib-openvswitch\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.912716 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovnkube-script-lib\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913104 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafc5e2d-4876-4e8a-9c9d-740a854045c2-env-overrides\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913159 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894b8144-59f5-4387-b59b-4564ea772cf7-serviceca\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913202 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-systemd\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913232 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-slash\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-systemd\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913543 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-host\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-run-ovn\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-etc-tuned\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913679 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218c03bf-de10-4fbe-88ba-91869f85e495-etc-kubernetes\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913683 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafc5e2d-4876-4e8a-9c9d-740a854045c2-host-kubelet\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913730 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-run-k8s-cni-cncf-io\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.914994 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913742 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5227952a-01f8-411a-870b-b208ca062c27-host-var-lib-cni-multus\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.916025 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.913801 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce04d9bb-c7f1-48ee-879a-32cac449f751-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.916025 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.914376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafc5e2d-4876-4e8a-9c9d-740a854045c2-ovn-node-metrics-cert\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.916310 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.916275 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:26.916310 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.916304 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:26.916438 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.916321 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:26.916438 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:26.916390 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:27.416368383 +0000 UTC m=+3.078162739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:26.917486 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.917461 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntmvc\" (UniqueName: \"kubernetes.io/projected/894b8144-59f5-4387-b59b-4564ea772cf7-kube-api-access-ntmvc\") pod \"node-ca-5kcqk\" (UID: \"894b8144-59f5-4387-b59b-4564ea772cf7\") " pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:26.917701 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.917681 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/218c03bf-de10-4fbe-88ba-91869f85e495-tmp\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.919055 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.918905 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w9kr\" (UniqueName: \"kubernetes.io/projected/5227952a-01f8-411a-870b-b208ca062c27-kube-api-access-2w9kr\") pod \"multus-f9kxd\" (UID: \"5227952a-01f8-411a-870b-b208ca062c27\") " pod="openshift-multus/multus-f9kxd" Jul 10 01:00:26.919691 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.919668 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpb4j\" (UniqueName: \"kubernetes.io/projected/ce04d9bb-c7f1-48ee-879a-32cac449f751-kube-api-access-lpb4j\") pod \"aws-ebs-csi-driver-node-d2bfs\" (UID: \"ce04d9bb-c7f1-48ee-879a-32cac449f751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:26.920353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.920328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h969f\" (UniqueName: \"kubernetes.io/projected/cf888257-f4f1-47c2-a54b-eba014c092b6-kube-api-access-h969f\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:26.926516 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.926492 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f54p\" (UniqueName: \"kubernetes.io/projected/bafc5e2d-4876-4e8a-9c9d-740a854045c2-kube-api-access-5f54p\") pod \"ovnkube-node-cqgp9\" (UID: \"bafc5e2d-4876-4e8a-9c9d-740a854045c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:26.927286 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.927259 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvpx5\" (UniqueName: \"kubernetes.io/projected/218c03bf-de10-4fbe-88ba-91869f85e495-kube-api-access-jvpx5\") pod \"tuned-hdwcf\" (UID: \"218c03bf-de10-4fbe-88ba-91869f85e495\") " pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:26.935194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.935160 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:26.944087 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.943991 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" event={"ID":"3e6349be74bf44ff9b94e695d15114e3","Type":"ContainerStarted","Data":"a23a6de4dc66811d3cf241bd8354a0f4ddbb379033a900ad736a7fbb1c14e225"} Jul 10 01:00:26.945900 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:26.945859 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" event={"ID":"78350d0908fbb8795cb0dc5d10a96325","Type":"ContainerStarted","Data":"3044aacaac7d3f94199bb6bfd735fde0c4e49b12e712c7129c360565512986c1"} Jul 10 01:00:27.008829 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008761 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91de7b74-be93-4945-9d06-c036c40e4f1b-agent-certs\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:27.008829 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1833f960-875b-4907-90c8-bcdb3c5ffb41-hosts-file\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91de7b74-be93-4945-9d06-c036c40e4f1b-konnectivity-ca\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008879 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3405dad4-9e04-4674-a806-2fd4e09b4966-host-slash\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-system-cni-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-kubelet-config\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.008975 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cnibin\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbmxb\" (UniqueName: \"kubernetes.io/projected/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-kube-api-access-pbmxb\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009061 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7247g\" (UniqueName: \"kubernetes.io/projected/1833f960-875b-4907-90c8-bcdb3c5ffb41-kube-api-access-7247g\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.009090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmkh\" (UniqueName: \"kubernetes.io/projected/3405dad4-9e04-4674-a806-2fd4e09b4966-kube-api-access-4bmkh\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3405dad4-9e04-4674-a806-2fd4e09b4966-iptables-alerter-script\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1833f960-875b-4907-90c8-bcdb3c5ffb41-tmp-dir\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-dbus\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.009411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009363 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-os-release\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.009677 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.009486 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-os-release\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.010194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010154 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.010323 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010256 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1833f960-875b-4907-90c8-bcdb3c5ffb41-hosts-file\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.010323 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010257 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-system-cni-dir\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.010434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010317 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3405dad4-9e04-4674-a806-2fd4e09b4966-host-slash\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.010434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-kubelet-config\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.010434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010410 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cnibin\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.010669 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010645 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3405dad4-9e04-4674-a806-2fd4e09b4966-iptables-alerter-script\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.010927 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010905 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.011048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91de7b74-be93-4945-9d06-c036c40e4f1b-konnectivity-ca\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:27.011048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.010929 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/389ecb05-4601-4cc5-a414-04cca57b42c0-dbus\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.011048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.011024 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.011048 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.011037 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:27.011267 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.011125 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:27.511103117 +0000 UTC m=+3.172897458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:27.011267 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.011192 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.011267 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.011221 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1833f960-875b-4907-90c8-bcdb3c5ffb41-tmp-dir\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.013903 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.013688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91de7b74-be93-4945-9d06-c036c40e4f1b-agent-certs\") pod \"konnectivity-agent-l7v52\" (UID: \"91de7b74-be93-4945-9d06-c036c40e4f1b\") " pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:27.022500 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.022466 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbmxb\" (UniqueName: \"kubernetes.io/projected/1e5f96c9-8b03-4bba-a377-5d43f5d90c50-kube-api-access-pbmxb\") pod \"multus-additional-cni-plugins-k8frj\" (UID: \"1e5f96c9-8b03-4bba-a377-5d43f5d90c50\") " pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.023302 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.023259 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmkh\" (UniqueName: \"kubernetes.io/projected/3405dad4-9e04-4674-a806-2fd4e09b4966-kube-api-access-4bmkh\") pod \"iptables-alerter-brfdr\" (UID: \"3405dad4-9e04-4674-a806-2fd4e09b4966\") " pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.023462 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.023412 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7247g\" (UniqueName: \"kubernetes.io/projected/1833f960-875b-4907-90c8-bcdb3c5ffb41-kube-api-access-7247g\") pod \"node-resolver-q9pks\" (UID: \"1833f960-875b-4907-90c8-bcdb3c5ffb41\") " pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.045969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.045934 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:27.092548 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.092503 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" Jul 10 01:00:27.105482 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.105440 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218c03bf_de10_4fbe_88ba_91869f85e495.slice/crio-46a9689fe03878695fadf8c5dc6c23bcbadfdf05f61acdf734161f6ce12c336e WatchSource:0}: Error finding container 46a9689fe03878695fadf8c5dc6c23bcbadfdf05f61acdf734161f6ce12c336e: Status 404 returned error can't find the container with id 46a9689fe03878695fadf8c5dc6c23bcbadfdf05f61acdf734161f6ce12c336e Jul 10 01:00:27.108580 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.108465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9kxd" Jul 10 01:00:27.118037 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.118011 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" Jul 10 01:00:27.119740 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.119708 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5227952a_01f8_411a_870b_b208ca062c27.slice/crio-bba78779f7fa7b132fc5c3e6f8437165576cd6b15e2539e0d1f632cbffa59126 WatchSource:0}: Error finding container bba78779f7fa7b132fc5c3e6f8437165576cd6b15e2539e0d1f632cbffa59126: Status 404 returned error can't find the container with id bba78779f7fa7b132fc5c3e6f8437165576cd6b15e2539e0d1f632cbffa59126 Jul 10 01:00:27.124706 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.124678 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5kcqk" Jul 10 01:00:27.130374 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.130345 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce04d9bb_c7f1_48ee_879a_32cac449f751.slice/crio-3b5afdffb7df8a348d639e28f286964f02a07a2e6b16bd758b554607955f5959 WatchSource:0}: Error finding container 3b5afdffb7df8a348d639e28f286964f02a07a2e6b16bd758b554607955f5959: Status 404 returned error can't find the container with id 3b5afdffb7df8a348d639e28f286964f02a07a2e6b16bd758b554607955f5959 Jul 10 01:00:27.132833 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.132615 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:27.138659 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.138618 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894b8144_59f5_4387_b59b_4564ea772cf7.slice/crio-c2ed2075a9f296f17934dcbfb689b036bdb5b8aacbfda546ccd046d94b71f018 WatchSource:0}: Error finding container c2ed2075a9f296f17934dcbfb689b036bdb5b8aacbfda546ccd046d94b71f018: Status 404 returned error can't find the container with id c2ed2075a9f296f17934dcbfb689b036bdb5b8aacbfda546ccd046d94b71f018 Jul 10 01:00:27.144833 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.144788 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafc5e2d_4876_4e8a_9c9d_740a854045c2.slice/crio-20585013bcb5367d15eadb85ea6a1bd7845fbe1bfa8c5c9f1acf72043ee2a7ed WatchSource:0}: Error finding container 20585013bcb5367d15eadb85ea6a1bd7845fbe1bfa8c5c9f1acf72043ee2a7ed: Status 404 returned error can't find the container with id 20585013bcb5367d15eadb85ea6a1bd7845fbe1bfa8c5c9f1acf72043ee2a7ed Jul 10 01:00:27.147553 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.147529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q9pks" Jul 10 01:00:27.157439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.157212 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8frj" Jul 10 01:00:27.159667 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.159634 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1833f960_875b_4907_90c8_bcdb3c5ffb41.slice/crio-a0e63d39973557f0ed09a6656ea8a3a3ed99b8cbba72b61606993fe322353403 WatchSource:0}: Error finding container a0e63d39973557f0ed09a6656ea8a3a3ed99b8cbba72b61606993fe322353403: Status 404 returned error can't find the container with id a0e63d39973557f0ed09a6656ea8a3a3ed99b8cbba72b61606993fe322353403 Jul 10 01:00:27.160255 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.160218 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:27.169371 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.169336 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5f96c9_8b03_4bba_a377_5d43f5d90c50.slice/crio-c15058bd6bcecbe31273d9c94e86e8b9377d04b7f334d25dea4a552384e46125 WatchSource:0}: Error finding container c15058bd6bcecbe31273d9c94e86e8b9377d04b7f334d25dea4a552384e46125: Status 404 returned error can't find the container with id c15058bd6bcecbe31273d9c94e86e8b9377d04b7f334d25dea4a552384e46125 Jul 10 01:00:27.172352 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.172318 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91de7b74_be93_4945_9d06_c036c40e4f1b.slice/crio-84df91918fd859a2d7cbdd0107cba20aa44cd9438218fdd58960ed37dddb9bc2 WatchSource:0}: Error finding container 84df91918fd859a2d7cbdd0107cba20aa44cd9438218fdd58960ed37dddb9bc2: Status 404 returned error can't find the container with id 84df91918fd859a2d7cbdd0107cba20aa44cd9438218fdd58960ed37dddb9bc2 Jul 10 01:00:27.172442 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.172371 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-brfdr" Jul 10 01:00:27.184284 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:00:27.184100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3405dad4_9e04_4674_a806_2fd4e09b4966.slice/crio-bbe24a787f02bd46ca0d04e9c6c27d102fa273b18db1bb142eda48c6665f477f WatchSource:0}: Error finding container bbe24a787f02bd46ca0d04e9c6c27d102fa273b18db1bb142eda48c6665f477f: Status 404 returned error can't find the container with id bbe24a787f02bd46ca0d04e9c6c27d102fa273b18db1bb142eda48c6665f477f Jul 10 01:00:27.412995 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.412953 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:27.413182 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.413154 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:27.413313 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.413295 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:28.413271852 +0000 UTC m=+4.075066205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:27.513740 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.513694 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:27.513966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.513793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:27.513966 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.513910 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:27.513966 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.513926 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:27.513966 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.513955 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:27.514148 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.513973 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:27.514148 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.514000 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:28.513983789 +0000 UTC m=+4.175778128 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:27.514148 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:27.514067 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:28.514013001 +0000 UTC m=+4.175807353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:27.828985 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.828841 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:55:27 +0000 UTC" deadline="2028-03-13 08:11:36.254828909 +0000 UTC" Jul 10 01:00:27.828985 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.828881 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14695h11m8.425951532s" Jul 10 01:00:27.948505 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.948470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"20585013bcb5367d15eadb85ea6a1bd7845fbe1bfa8c5c9f1acf72043ee2a7ed"} Jul 10 01:00:27.949706 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.949678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5kcqk" event={"ID":"894b8144-59f5-4387-b59b-4564ea772cf7","Type":"ContainerStarted","Data":"c2ed2075a9f296f17934dcbfb689b036bdb5b8aacbfda546ccd046d94b71f018"} Jul 10 01:00:27.950775 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.950746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" event={"ID":"ce04d9bb-c7f1-48ee-879a-32cac449f751","Type":"ContainerStarted","Data":"3b5afdffb7df8a348d639e28f286964f02a07a2e6b16bd758b554607955f5959"} Jul 10 01:00:27.952022 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.951993 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" event={"ID":"218c03bf-de10-4fbe-88ba-91869f85e495","Type":"ContainerStarted","Data":"46a9689fe03878695fadf8c5dc6c23bcbadfdf05f61acdf734161f6ce12c336e"} Jul 10 01:00:27.953232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.953198 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-brfdr" event={"ID":"3405dad4-9e04-4674-a806-2fd4e09b4966","Type":"ContainerStarted","Data":"bbe24a787f02bd46ca0d04e9c6c27d102fa273b18db1bb142eda48c6665f477f"} Jul 10 01:00:27.954698 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.954668 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l7v52" event={"ID":"91de7b74-be93-4945-9d06-c036c40e4f1b","Type":"ContainerStarted","Data":"84df91918fd859a2d7cbdd0107cba20aa44cd9438218fdd58960ed37dddb9bc2"} Jul 10 01:00:27.955946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.955914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerStarted","Data":"c15058bd6bcecbe31273d9c94e86e8b9377d04b7f334d25dea4a552384e46125"} Jul 10 01:00:27.957047 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.957025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q9pks" event={"ID":"1833f960-875b-4907-90c8-bcdb3c5ffb41","Type":"ContainerStarted","Data":"a0e63d39973557f0ed09a6656ea8a3a3ed99b8cbba72b61606993fe322353403"} Jul 10 01:00:27.958232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:27.958211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9kxd" event={"ID":"5227952a-01f8-411a-870b-b208ca062c27","Type":"ContainerStarted","Data":"bba78779f7fa7b132fc5c3e6f8437165576cd6b15e2539e0d1f632cbffa59126"} Jul 10 01:00:28.422769 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.422734 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:28.422986 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.422949 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:28.423120 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.423033 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:30.423011819 +0000 UTC m=+6.084806167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:28.524087 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.524036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:28.524275 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.524108 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:28.524352 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524288 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:28.524352 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524313 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:28.524352 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524328 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:28.524496 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524397 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:30.52437406 +0000 UTC m=+6.186168398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:28.524858 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524835 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:28.524993 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.524895 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:30.524877536 +0000 UTC m=+6.186671877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:28.778477 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.778388 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 01:00:28.829285 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.829211 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:55:27 +0000 UTC" deadline="2028-04-09 10:14:48.396832936 +0000 UTC" Jul 10 01:00:28.829285 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.829282 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15345h14m19.567554854s" Jul 10 01:00:28.944551 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.944511 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:28.944735 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.944668 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:28.945592 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.945143 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:28.945592 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.945259 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:28.945592 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.945328 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:28.945592 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:28.945400 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:28.996016 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:28.995921 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" event={"ID":"78350d0908fbb8795cb0dc5d10a96325","Type":"ContainerStarted","Data":"f35499766d5c700c606e7572018cdb5b918c2bac1f1a92c1d30b01a19e40ea58"} Jul 10 01:00:30.005739 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.004969 2565 generic.go:356] "Generic (PLEG): container finished" podID="3e6349be74bf44ff9b94e695d15114e3" containerID="08a652e665ff5844c30ec98876e6c0164505b76cdfb41bd687859ea84778e4ce" exitCode=0 Jul 10 01:00:30.005739 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.005079 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" event={"ID":"3e6349be74bf44ff9b94e695d15114e3","Type":"ContainerDied","Data":"08a652e665ff5844c30ec98876e6c0164505b76cdfb41bd687859ea84778e4ce"} Jul 10 01:00:30.021156 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.021092 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-136.ec2.internal" podStartSLOduration=4.021066375 podStartE2EDuration="4.021066375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:29.012688637 +0000 UTC m=+4.674482996" watchObservedRunningTime="2026-07-10 01:00:30.021066375 +0000 UTC m=+5.682860738" Jul 10 01:00:30.445776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.445737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:30.446004 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.445928 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:30.446004 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.445992 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:34.445973361 +0000 UTC m=+10.107767696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:30.547039 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.546999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:30.547229 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.547094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:30.547229 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547216 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:30.547360 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547278 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:34.547257888 +0000 UTC m=+10.209052263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:30.547787 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547766 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:30.547910 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547795 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:30.547910 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547808 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:30.547910 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.547855 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:34.547839903 +0000 UTC m=+10.209634239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:30.940348 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.940312 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:30.940549 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.940464 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:30.940857 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.940835 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:30.940977 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.940955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:30.941043 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:30.941013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:30.941118 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:30.941099 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:32.938519 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:32.938484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:32.939028 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:32.938673 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:32.939289 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:32.939264 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:32.939576 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:32.939551 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:32.940078 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:32.940060 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:32.940362 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:32.940328 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:34.477717 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.477679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:34.478207 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.477807 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:34.478207 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.477862 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:42.477848724 +0000 UTC m=+18.139643056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:34.578602 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.578559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:34.578767 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.578657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:34.578767 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578757 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:34.578872 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578780 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:34.578872 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578786 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:34.578872 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578805 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:34.578872 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578837 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:42.578819356 +0000 UTC m=+18.240613691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:34.578872 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.578860 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:42.578843782 +0000 UTC m=+18.240638132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:34.938110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.938071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:34.938283 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.938173 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:34.938283 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:34.938207 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:34.938283 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.938221 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:34.938444 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.938294 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:34.938444 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:34.938402 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:36.940121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:36.940087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:36.940577 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:36.940223 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:36.940659 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:36.940602 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:36.940710 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:36.940695 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:36.940763 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:36.940744 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:36.940855 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:36.940822 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:38.937707 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:38.937466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:38.938293 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:38.937466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:38.938293 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:38.937767 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:38.938293 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:38.937544 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:38.938293 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:38.937842 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:38.938293 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:38.937958 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:39.028536 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.028482 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" event={"ID":"ce04d9bb-c7f1-48ee-879a-32cac449f751","Type":"ContainerStarted","Data":"2b9c65533a9986730c4aa85d598a04ee7e1be08560787eab526c3b23d6aa64af"} Jul 10 01:00:39.030750 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.030719 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" event={"ID":"3e6349be74bf44ff9b94e695d15114e3","Type":"ContainerStarted","Data":"cbf2f8c543a07713921d2020283987140d738b56992658cb3fa625ef833e1616"} Jul 10 01:00:39.032957 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.032921 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l7v52" event={"ID":"91de7b74-be93-4945-9d06-c036c40e4f1b","Type":"ContainerStarted","Data":"701b08ca5599ce36c52b4069d5f6270fd30ceb29147970878f51458febdbd3bc"} Jul 10 01:00:39.035161 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.035134 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerStarted","Data":"3a038028f222664f48e3820dee6c889cde28414fdbf1d8c197463a922f0f0950"} Jul 10 01:00:39.036982 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.036951 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q9pks" event={"ID":"1833f960-875b-4907-90c8-bcdb3c5ffb41","Type":"ContainerStarted","Data":"e99b0fcad45985d5d84ad78aef9ebac3c5019e69ff618f028fbce06588528723"} Jul 10 01:00:39.038585 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.038559 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5kcqk" event={"ID":"894b8144-59f5-4387-b59b-4564ea772cf7","Type":"ContainerStarted","Data":"a81b88e7285713b66bd113dc782368f8b55d348069d96ec94655fe2de39d98bd"} Jul 10 01:00:39.045464 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.045406 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-136.ec2.internal" podStartSLOduration=13.04538735 podStartE2EDuration="13.04538735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:39.045081242 +0000 UTC m=+14.706875598" watchObservedRunningTime="2026-07-10 01:00:39.04538735 +0000 UTC m=+14.707181714" Jul 10 01:00:39.058724 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.058665 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5kcqk" podStartSLOduration=2.791786945 podStartE2EDuration="14.058650445s" totalImagesPullingTime="11.2668635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:39.058149621 +0000 UTC m=+14.719943979" watchObservedRunningTime="2026-07-10 01:00:39.058650445 +0000 UTC m=+14.720444796" Jul 10 01:00:39.071580 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.071497 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q9pks" podStartSLOduration=2.791597475 podStartE2EDuration="14.071478116s" totalImagesPullingTime="11.279880641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:39.071441507 +0000 UTC m=+14.733235872" watchObservedRunningTime="2026-07-10 01:00:39.071478116 +0000 UTC m=+14.733272474" Jul 10 01:00:39.110579 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.110520 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l7v52" podStartSLOduration=2.870574749 podStartE2EDuration="14.110498961s" totalImagesPullingTime="11.239924212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:39.11046696 +0000 UTC m=+14.772261318" watchObservedRunningTime="2026-07-10 01:00:39.110498961 +0000 UTC m=+14.772293311" Jul 10 01:00:39.597702 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.597667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:39.598333 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:39.598309 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:40.938105 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:40.938066 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:40.938816 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:40.938071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:40.938816 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:40.938202 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:40.938816 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:40.938280 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:40.938816 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:40.938068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:40.938816 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:40.938372 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:41.045372 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:41.044968 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 01:00:41.045372 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:41.044995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-brfdr" event={"ID":"3405dad4-9e04-4674-a806-2fd4e09b4966","Type":"ContainerStarted","Data":"474464431f26c16a07594f5eba17245bc350958a66e7738172a119486561f5df"} Jul 10 01:00:41.058861 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:41.058798 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-brfdr" podStartSLOduration=4.805165055 podStartE2EDuration="16.05877807s" totalImagesPullingTime="11.253613015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:41.058425867 +0000 UTC m=+16.720220222" watchObservedRunningTime="2026-07-10 01:00:41.05877807 +0000 UTC m=+16.720572443" Jul 10 01:00:42.542717 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.542672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:42.543163 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.542847 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:42.543163 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.542934 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:58.542914391 +0000 UTC m=+34.204708725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:42.644086 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.644045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:42.644277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.644123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:42.644277 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644239 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:42.644277 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644262 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:42.644461 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644286 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:42.644461 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644300 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:42.644461 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644326 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:58.644295476 +0000 UTC m=+34.306089833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:42.644461 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.644358 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:00:58.644342186 +0000 UTC m=+34.306136533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:42.938142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.938092 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:42.938313 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.938151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:42.938313 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.938226 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:42.938431 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:42.938323 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:42.938431 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.938417 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:42.938525 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:42.938507 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:43.051232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:43.051173 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="3a038028f222664f48e3820dee6c889cde28414fdbf1d8c197463a922f0f0950" exitCode=0 Jul 10 01:00:43.051232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:43.051205 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"3a038028f222664f48e3820dee6c889cde28414fdbf1d8c197463a922f0f0950"} Jul 10 01:00:44.937903 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:44.937862 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:44.938535 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:44.937992 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:44.938535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:44.938400 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:44.938535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:44.938493 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:44.938698 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:44.938524 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:44.938698 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:44.938574 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:45.839675 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:45.839630 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:45.839872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:45.839838 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 01:00:45.840577 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:45.840468 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l7v52" Jul 10 01:00:46.938089 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:46.938052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:46.938660 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:46.938052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:46.938660 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:46.938181 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:46.938660 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:46.938178 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:46.938660 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:46.938282 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:46.938660 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:46.938370 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:47.576642 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:47.576519 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 01:00:47.865483 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:47.865133 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T01:00:47.576632556Z","UUID":"fe28e079-2992-459a-aa58-4838049fde45","Handler":null,"Name":"","Endpoint":""} Jul 10 01:00:47.866847 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:47.866819 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 01:00:47.867014 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:47.866869 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 01:00:48.065510 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.065430 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:00:48.066142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.065839 2565 generic.go:356] "Generic (PLEG): container finished" podID="bafc5e2d-4876-4e8a-9c9d-740a854045c2" containerID="2b8b7bb2f44a47d33ea3148aeb6b212f0f8d067c53260b4a4df0b15c5f47e1c7" exitCode=1 Jul 10 01:00:48.066142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.065916 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"f4daf88fda4f307c04077eef3626b6c58dce96bad2af911ffed003b697f764f3"} Jul 10 01:00:48.066142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.066036 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"da1ed3dce9a93f5d49313f6d95d0c428158f1c68997372657538962266429087"} Jul 10 01:00:48.066142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.066092 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"d32249dbd0bda616c0c0f9fae3b754943ba2db45fa90c4e42081844705258c6e"} Jul 10 01:00:48.066142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.066139 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"12effe2e417feb7a6fa6afc59dafb28b976d4902a099265afabaf4127bebae14"} Jul 10 01:00:48.066371 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.066187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerDied","Data":"2b8b7bb2f44a47d33ea3148aeb6b212f0f8d067c53260b4a4df0b15c5f47e1c7"} Jul 10 01:00:48.066371 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.066239 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"e7cd413d9707fe6b834161bf9afcc3a4105956e1a078ef71d601f89566257650"} Jul 10 01:00:48.067945 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.067918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" event={"ID":"ce04d9bb-c7f1-48ee-879a-32cac449f751","Type":"ContainerStarted","Data":"4cc87422299cccc484297dfe548f05647f83bef8ef2d17d047a9426b09c4984d"} Jul 10 01:00:48.069484 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.069457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" event={"ID":"218c03bf-de10-4fbe-88ba-91869f85e495","Type":"ContainerStarted","Data":"faae7bc1ebf5fc5d052ca399cfd35aee80806e3998010065ec460d9e6392dbb0"} Jul 10 01:00:48.071242 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.071217 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9kxd" event={"ID":"5227952a-01f8-411a-870b-b208ca062c27","Type":"ContainerStarted","Data":"22fcf809c769e991f3180a5fe60801a96b920e2afd1f188a5994f1596f428a34"} Jul 10 01:00:48.088143 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.088097 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hdwcf" podStartSLOduration=11.704259674 podStartE2EDuration="23.08808317s" totalImagesPullingTime="11.383823496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:48.088043952 +0000 UTC m=+23.749838307" watchObservedRunningTime="2026-07-10 01:00:48.08808317 +0000 UTC m=+23.749877525" Jul 10 01:00:48.104393 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.104345 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f9kxd" podStartSLOduration=2.938094729 podStartE2EDuration="23.104329945s" totalImagesPullingTime="20.166235216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:48.104086207 +0000 UTC m=+23.765880561" watchObservedRunningTime="2026-07-10 01:00:48.104329945 +0000 UTC m=+23.766124301" Jul 10 01:00:48.937920 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.937869 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:48.938139 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.937936 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:48.938139 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:48.937869 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:48.938139 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:48.938027 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:48.938139 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:48.938096 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:48.938335 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:48.938193 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:50.079961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.079931 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:00:50.080560 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.080531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"d2984598d6c17ae33ac210c1b699a43e10d915f3980b8df5aa0fb1bbcacc0faa"} Jul 10 01:00:50.086845 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.086755 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" event={"ID":"ce04d9bb-c7f1-48ee-879a-32cac449f751","Type":"ContainerStarted","Data":"bfed65b41fa5cc2dad5ab318fb1969197e59e8f7b12821a62c982b68d30b1963"} Jul 10 01:00:50.103515 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.103457 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bfs" podStartSLOduration=3.397980604 podStartE2EDuration="25.103437929s" totalImagesPullingTime="21.705457325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:50.101998916 +0000 UTC m=+25.763793271" watchObservedRunningTime="2026-07-10 01:00:50.103437929 +0000 UTC m=+25.765232329" Jul 10 01:00:50.937512 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.937471 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:50.937686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.937482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:50.937686 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:50.937615 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:50.937686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:50.937636 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:50.937869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:50.937718 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:50.937869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:50.937815 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:52.937624 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:52.937539 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:52.938113 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:52.937549 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:52.938113 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:52.937668 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:52.938113 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:52.937661 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:52.938113 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:52.937754 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:52.938113 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:52.937854 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:53.097422 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.097394 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:00:53.097836 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.097804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"7d2ad7bc5747504e301732399743990146fd9a44bc1c57c9f3875e5e9215868f"} Jul 10 01:00:53.098045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.098026 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:53.098258 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.098239 2565 scope.go:117] "RemoveContainer" containerID="2b8b7bb2f44a47d33ea3148aeb6b212f0f8d067c53260b4a4df0b15c5f47e1c7" Jul 10 01:00:53.099535 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.099515 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="94131c2abb416fc243070d4f76dcc8a93a1da7c61a9050d72cc22e9cab6ee968" exitCode=0 Jul 10 01:00:53.099614 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.099550 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"94131c2abb416fc243070d4f76dcc8a93a1da7c61a9050d72cc22e9cab6ee968"} Jul 10 01:00:53.116962 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:53.116933 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:54.106110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.106085 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:00:54.106577 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.106480 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" event={"ID":"bafc5e2d-4876-4e8a-9c9d-740a854045c2","Type":"ContainerStarted","Data":"aa4c002301fa8d5f8e7b6accc86ffa500c904a57cbcd855adb1ed6f71a08b2b5"} Jul 10 01:00:54.106764 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.106741 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:54.106971 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.106953 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:54.124967 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.124937 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:00:54.134299 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.134253 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" podStartSLOduration=8.926910346 podStartE2EDuration="29.13423455s" totalImagesPullingTime="20.207324204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:54.132700598 +0000 UTC m=+29.794494957" watchObservedRunningTime="2026-07-10 01:00:54.13423455 +0000 UTC m=+29.796028904" Jul 10 01:00:54.393267 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.393024 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5cp6"] Jul 10 01:00:54.393415 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.393358 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:54.393503 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:54.393475 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:54.396257 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.396219 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rztvp"] Jul 10 01:00:54.396417 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.396389 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:54.396543 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:54.396521 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:54.396803 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.396778 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2xjk"] Jul 10 01:00:54.396906 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:54.396877 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:54.397003 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:54.396982 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:55.110764 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:55.110729 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="df4f8d239d3cac1e662bcf26fcd2d885bc87c01776d06e29fccf1532ef41c046" exitCode=0 Jul 10 01:00:55.111284 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:55.110798 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"df4f8d239d3cac1e662bcf26fcd2d885bc87c01776d06e29fccf1532ef41c046"} Jul 10 01:00:55.937903 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:55.937847 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:55.938073 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:55.937866 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:55.938073 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:55.938011 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:55.938187 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:55.937982 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:55.938187 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:55.938098 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:55.938187 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:55.938146 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:57.118427 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:57.118395 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="870155469aba28357c7d62fa5cd30313885fe6a623f6e302bff1451cf2b50537" exitCode=0 Jul 10 01:00:57.119054 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:57.118444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"870155469aba28357c7d62fa5cd30313885fe6a623f6e302bff1451cf2b50537"} Jul 10 01:00:57.937772 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:57.937728 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:57.937997 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:57.937736 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:57.937997 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:57.937864 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:57.937997 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:57.937962 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:00:57.937997 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:57.937744 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:57.938206 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:57.938036 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:58.558463 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:58.558419 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:58.558961 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.558578 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:58.558961 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.558652 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:01:30.558634976 +0000 UTC m=+66.220429383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 01:00:58.659515 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:58.659467 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:58.659552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659635 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659666 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659674 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659688 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6gsxd for pod openshift-network-diagnostics/network-check-target-k2xjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659729 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret podName:389ecb05-4601-4cc5-a414-04cca57b42c0 nodeName:}" failed. No retries permitted until 2026-07-10 01:01:30.659709189 +0000 UTC m=+66.321503523 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret") pod "global-pull-secret-syncer-x5cp6" (UID: "389ecb05-4601-4cc5-a414-04cca57b42c0") : object "kube-system"/"original-pull-secret" not registered Jul 10 01:00:58.659774 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:58.659745 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd podName:050c382d-d5c4-45ea-9cd5-4291a40cf140 nodeName:}" failed. No retries permitted until 2026-07-10 01:01:30.659738203 +0000 UTC m=+66.321532535 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6gsxd" (UniqueName: "kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd") pod "network-check-target-k2xjk" (UID: "050c382d-d5c4-45ea-9cd5-4291a40cf140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 01:00:59.937955 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:59.937916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:00:59.938430 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:59.937916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:00:59.938430 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:59.938053 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5cp6" podUID="389ecb05-4601-4cc5-a414-04cca57b42c0" Jul 10 01:00:59.938430 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:00:59.937916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:00:59.938430 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:59.938150 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:00:59.938430 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:00:59.938194 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2xjk" podUID="050c382d-d5c4-45ea-9cd5-4291a40cf140" Jul 10 01:01:00.107546 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.107508 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-136.ec2.internal" event="NodeReady" Jul 10 01:01:00.107709 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.107638 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 01:01:00.159960 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.158836 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nvklj"] Jul 10 01:01:00.181477 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.181434 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wbxdj"] Jul 10 01:01:00.181679 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.181447 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.184274 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.184247 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 01:01:00.184431 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.184337 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 01:01:00.184753 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.184734 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bmxj4\"" Jul 10 01:01:00.197996 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.197942 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbxdj"] Jul 10 01:01:00.198152 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.198009 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvklj"] Jul 10 01:01:00.198152 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.198041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.200509 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.200483 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 01:01:00.200633 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.200483 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-86zlv\"" Jul 10 01:01:00.200633 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.200482 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 01:01:00.200721 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.200673 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 01:01:00.269742 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.269939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269772 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljxwh\" (UniqueName: \"kubernetes.io/projected/e1e570c0-1149-4943-bffc-5d2f0f8c216d-kube-api-access-ljxwh\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.269939 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e570c0-1149-4943-bffc-5d2f0f8c216d-tmp-dir\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.270045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e570c0-1149-4943-bffc-5d2f0f8c216d-config-volume\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.270045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-cert\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.270045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.269981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47xhq\" (UniqueName: \"kubernetes.io/projected/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-kube-api-access-47xhq\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.370760 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e570c0-1149-4943-bffc-5d2f0f8c216d-tmp-dir\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.370760 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370743 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e570c0-1149-4943-bffc-5d2f0f8c216d-config-volume\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.370986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370772 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-cert\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.370986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370808 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47xhq\" (UniqueName: \"kubernetes.io/projected/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-kube-api-access-47xhq\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.370986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.370986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.370876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljxwh\" (UniqueName: \"kubernetes.io/projected/e1e570c0-1149-4943-bffc-5d2f0f8c216d-kube-api-access-ljxwh\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.371131 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.371097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e570c0-1149-4943-bffc-5d2f0f8c216d-tmp-dir\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.371462 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:00.371436 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:00.371595 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:00.371510 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:00.871487939 +0000 UTC m=+36.533282273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:00.372355 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.372189 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e570c0-1149-4943-bffc-5d2f0f8c216d-config-volume\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.376718 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.376688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-cert\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.380541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.380510 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljxwh\" (UniqueName: \"kubernetes.io/projected/e1e570c0-1149-4943-bffc-5d2f0f8c216d-kube-api-access-ljxwh\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.380700 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.380684 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47xhq\" (UniqueName: \"kubernetes.io/projected/6ab167a5-1592-4033-ba8e-d9c05b9d17d1-kube-api-access-47xhq\") pod \"ingress-canary-wbxdj\" (UID: \"6ab167a5-1592-4033-ba8e-d9c05b9d17d1\") " pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.507835 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.507793 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbxdj" Jul 10 01:01:00.703243 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.703208 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbxdj"] Jul 10 01:01:00.708922 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:01:00.707708 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab167a5_1592_4033_ba8e_d9c05b9d17d1.slice/crio-8b87323f8a1359be36cc5a6a2a6ca3f11634a24045ca53c412a985027dae04f7 WatchSource:0}: Error finding container 8b87323f8a1359be36cc5a6a2a6ca3f11634a24045ca53c412a985027dae04f7: Status 404 returned error can't find the container with id 8b87323f8a1359be36cc5a6a2a6ca3f11634a24045ca53c412a985027dae04f7 Jul 10 01:01:00.875403 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:00.875357 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:00.875586 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:00.875513 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:00.875640 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:00.875590 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:01.875572256 +0000 UTC m=+37.537366593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:01.128475 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.128435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbxdj" event={"ID":"6ab167a5-1592-4033-ba8e-d9c05b9d17d1","Type":"ContainerStarted","Data":"8b87323f8a1359be36cc5a6a2a6ca3f11634a24045ca53c412a985027dae04f7"} Jul 10 01:01:01.883861 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.883819 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:01.884073 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:01.884031 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:01.884131 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:01.884106 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:03.884073484 +0000 UTC m=+39.545867842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:01.937707 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.937669 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:01:01.937930 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.937756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:01:01.937930 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.937780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:01:01.942961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942332 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 01:01:01.942961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942592 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 01:01:01.942961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942785 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 01:01:01.942961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942876 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjnvz\"" Jul 10 01:01:01.942961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942944 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 01:01:01.943373 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:01.942966 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dbd7q\"" Jul 10 01:01:03.899573 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:03.898981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:03.899573 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:03.899119 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:03.899573 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:03.899199 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:07.899175027 +0000 UTC m=+43.560969377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:05.140271 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:05.140094 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="bfbb6ea3b8e531ca518ec278bd2dfe5e68409f3b7ade70a39a65c6c4eba0393f" exitCode=0 Jul 10 01:01:05.140790 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:05.140176 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"bfbb6ea3b8e531ca518ec278bd2dfe5e68409f3b7ade70a39a65c6c4eba0393f"} Jul 10 01:01:05.141739 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:05.141717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbxdj" event={"ID":"6ab167a5-1592-4033-ba8e-d9c05b9d17d1","Type":"ContainerStarted","Data":"0b555e45a5f2a37fb5a2d083f209bf7838cd6a0ac3b40189998490c5cbdd4b93"} Jul 10 01:01:05.178739 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:05.178670 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wbxdj" podStartSLOduration=1.706600666 podStartE2EDuration="5.178647138s" totalImagesPullingTime="3.472046472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:05.177766574 +0000 UTC m=+40.839560929" watchObservedRunningTime="2026-07-10 01:01:05.178647138 +0000 UTC m=+40.840441496" Jul 10 01:01:06.147654 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:06.147620 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e5f96c9-8b03-4bba-a377-5d43f5d90c50" containerID="d857ecc30641205af2dc9b66f52675e33af2e04facd21450f3d9dc55a34f35ca" exitCode=0 Jul 10 01:01:06.148062 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:06.147685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerDied","Data":"d857ecc30641205af2dc9b66f52675e33af2e04facd21450f3d9dc55a34f35ca"} Jul 10 01:01:07.153541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:07.153509 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8frj" event={"ID":"1e5f96c9-8b03-4bba-a377-5d43f5d90c50","Type":"ContainerStarted","Data":"25d561b26129e396a2a6d5583111382ee8f02a88b12eef8712d25fa57f5197cb"} Jul 10 01:01:07.175773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:07.175655 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k8frj" podStartSLOduration=11.863154301 podStartE2EDuration="42.175637094s" totalImagesPullingTime="30.312482793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:07.174429921 +0000 UTC m=+42.836224276" watchObservedRunningTime="2026-07-10 01:01:07.175637094 +0000 UTC m=+42.837431448" Jul 10 01:01:07.931007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:07.930943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:07.931172 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:07.931107 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:07.931231 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:07.931174 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:15.931156377 +0000 UTC m=+51.592950711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:15.986910 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:15.986845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:15.987281 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:15.987016 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:15.987281 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:15.987095 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:01:31.987079511 +0000 UTC m=+67.648873844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:26.125192 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:26.125163 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqgp9" Jul 10 01:01:30.592029 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.591989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:01:30.594918 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.594875 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 01:01:30.603216 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:30.603181 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 01:01:30.603330 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:30.603268 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:02:34.603246922 +0000 UTC m=+130.265041254 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : secret "metrics-daemon-secret" not found Jul 10 01:01:30.692390 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.692358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:01:30.692529 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.692418 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:01:30.695121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.695100 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 01:01:30.695229 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.695211 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 01:01:30.705021 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.704998 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 01:01:30.706148 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.706125 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/389ecb05-4601-4cc5-a414-04cca57b42c0-original-pull-secret\") pod \"global-pull-secret-syncer-x5cp6\" (UID: \"389ecb05-4601-4cc5-a414-04cca57b42c0\") " pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:01:30.715652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.715622 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gsxd\" (UniqueName: \"kubernetes.io/projected/050c382d-d5c4-45ea-9cd5-4291a40cf140-kube-api-access-6gsxd\") pod \"network-check-target-k2xjk\" (UID: \"050c382d-d5c4-45ea-9cd5-4291a40cf140\") " pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:01:30.761193 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.761153 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5cp6" Jul 10 01:01:30.770718 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.770691 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dbd7q\"" Jul 10 01:01:30.778193 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.778162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:01:30.892496 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.892467 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5cp6"] Jul 10 01:01:30.896230 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:01:30.896184 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389ecb05_4601_4cc5_a414_04cca57b42c0.slice/crio-9d8f4a435a100cbfc88b1840c6d306ab82103581463cc10f4f5d20c193170fc7 WatchSource:0}: Error finding container 9d8f4a435a100cbfc88b1840c6d306ab82103581463cc10f4f5d20c193170fc7: Status 404 returned error can't find the container with id 9d8f4a435a100cbfc88b1840c6d306ab82103581463cc10f4f5d20c193170fc7 Jul 10 01:01:30.912584 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:30.912556 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2xjk"] Jul 10 01:01:30.916364 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:01:30.916330 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050c382d_d5c4_45ea_9cd5_4291a40cf140.slice/crio-5da9c4eed135825393b82462084cd108210e976a6159c225c2611ae7d5980946 WatchSource:0}: Error finding container 5da9c4eed135825393b82462084cd108210e976a6159c225c2611ae7d5980946: Status 404 returned error can't find the container with id 5da9c4eed135825393b82462084cd108210e976a6159c225c2611ae7d5980946 Jul 10 01:01:31.213599 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:31.213505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5cp6" event={"ID":"389ecb05-4601-4cc5-a414-04cca57b42c0","Type":"ContainerStarted","Data":"9d8f4a435a100cbfc88b1840c6d306ab82103581463cc10f4f5d20c193170fc7"} Jul 10 01:01:31.214718 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:31.214689 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2xjk" event={"ID":"050c382d-d5c4-45ea-9cd5-4291a40cf140","Type":"ContainerStarted","Data":"5da9c4eed135825393b82462084cd108210e976a6159c225c2611ae7d5980946"} Jul 10 01:01:32.003029 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:32.002992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:01:32.003524 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:32.003141 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:01:32.003524 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:01:32.003205 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:02:04.00318939 +0000 UTC m=+99.664983727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:01:36.230104 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:36.230057 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5cp6" event={"ID":"389ecb05-4601-4cc5-a414-04cca57b42c0","Type":"ContainerStarted","Data":"a1531ac4e1c9b3a2e95c4ea3a8a33c0142e6a9e9a54347d77b805f06cf734be8"} Jul 10 01:01:36.231484 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:36.231458 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2xjk" event={"ID":"050c382d-d5c4-45ea-9cd5-4291a40cf140","Type":"ContainerStarted","Data":"6bda214e29d07f664453ae1017917e428fec3b417ebd38b5f65d72af0a8b6e72"} Jul 10 01:01:36.231597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:36.231581 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:01:36.243672 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:36.243622 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x5cp6" podStartSLOduration=66.546695525 podStartE2EDuration="1m11.243607911s" totalImagesPullingTime="4.696912386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:36.243102038 +0000 UTC m=+71.904896393" watchObservedRunningTime="2026-07-10 01:01:36.243607911 +0000 UTC m=+71.905402267" Jul 10 01:01:36.256380 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:36.256320 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k2xjk" podStartSLOduration=66.582067133 podStartE2EDuration="1m11.256304182s" totalImagesPullingTime="4.674237049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:36.255856516 +0000 UTC m=+71.917650871" watchObservedRunningTime="2026-07-10 01:01:36.256304182 +0000 UTC m=+71.918098567" Jul 10 01:01:37.336991 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.336958 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp"] Jul 10 01:01:37.339152 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.339134 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.341541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.341517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 01:01:37.342721 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.342690 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 01:01:37.342721 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.342718 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 01:01:37.342919 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.342769 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 01:01:37.346999 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.346849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-tmp\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.346999 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.346938 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-klusterlet-config\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.347175 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.347005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf2r2\" (UniqueName: \"kubernetes.io/projected/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-kube-api-access-gf2r2\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.349565 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.349490 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp"] Jul 10 01:01:37.351734 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.351708 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d"] Jul 10 01:01:37.353654 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.353632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.357234 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.357207 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 01:01:37.357407 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.357290 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 01:01:37.357407 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.357332 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 01:01:37.357407 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.357372 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 01:01:37.365027 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.364996 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d"] Jul 10 01:01:37.447300 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447251 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447308 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447359 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dnn\" (UniqueName: \"kubernetes.io/projected/16dde9a7-f045-4df3-97b3-6b1f1259c7db-kube-api-access-h2dnn\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-tmp\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.447493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dde9a7-f045-4df3-97b3-6b1f1259c7db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447710 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-klusterlet-config\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.447710 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf2r2\" (UniqueName: \"kubernetes.io/projected/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-kube-api-access-gf2r2\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.447710 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447576 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.447847 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.447779 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-tmp\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.450079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.450049 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-klusterlet-config\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.456015 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.455992 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf2r2\" (UniqueName: \"kubernetes.io/projected/d95b5a8d-0fc7-46d1-9148-f003b6b25d0f-kube-api-access-gf2r2\") pod \"klusterlet-addon-workmgr-667b76d967-bcgsp\" (UID: \"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.548013 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.547979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548013 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548019 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548263 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548263 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548263 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dnn\" (UniqueName: \"kubernetes.io/projected/16dde9a7-f045-4df3-97b3-6b1f1259c7db-kube-api-access-h2dnn\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548263 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dde9a7-f045-4df3-97b3-6b1f1259c7db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.548917 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.548824 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dde9a7-f045-4df3-97b3-6b1f1259c7db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.550785 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.550753 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.550872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.550755 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-ca\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.551088 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.551070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.551293 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.551271 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16dde9a7-f045-4df3-97b3-6b1f1259c7db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.557817 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.557795 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dnn\" (UniqueName: \"kubernetes.io/projected/16dde9a7-f045-4df3-97b3-6b1f1259c7db-kube-api-access-h2dnn\") pod \"cluster-proxy-proxy-agent-b9877b597-n6d6d\" (UID: \"16dde9a7-f045-4df3-97b3-6b1f1259c7db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.653012 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.652927 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:37.673422 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.673386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:01:37.792630 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.792598 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp"] Jul 10 01:01:37.796315 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:01:37.796284 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95b5a8d_0fc7_46d1_9148_f003b6b25d0f.slice/crio-dc5405516346f4d200b170c59772d5660de46205e0f02b6eb5d0554227a9f2f3 WatchSource:0}: Error finding container dc5405516346f4d200b170c59772d5660de46205e0f02b6eb5d0554227a9f2f3: Status 404 returned error can't find the container with id dc5405516346f4d200b170c59772d5660de46205e0f02b6eb5d0554227a9f2f3 Jul 10 01:01:37.813317 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:37.813287 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d"] Jul 10 01:01:37.816535 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:01:37.816505 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16dde9a7_f045_4df3_97b3_6b1f1259c7db.slice/crio-abc46ad0b5895bbe74e988dcb1ef5de08754e51eb5717e05e954fb1093414baf WatchSource:0}: Error finding container abc46ad0b5895bbe74e988dcb1ef5de08754e51eb5717e05e954fb1093414baf: Status 404 returned error can't find the container with id abc46ad0b5895bbe74e988dcb1ef5de08754e51eb5717e05e954fb1093414baf Jul 10 01:01:38.237997 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:38.237955 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" event={"ID":"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f","Type":"ContainerStarted","Data":"dc5405516346f4d200b170c59772d5660de46205e0f02b6eb5d0554227a9f2f3"} Jul 10 01:01:38.239048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:38.239022 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerStarted","Data":"abc46ad0b5895bbe74e988dcb1ef5de08754e51eb5717e05e954fb1093414baf"} Jul 10 01:01:43.255908 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.255715 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" event={"ID":"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f","Type":"ContainerStarted","Data":"2885a91de69f5fb6186f4f39c54dc913f7b4f38d5990d151dd76a97975ac47e8"} Jul 10 01:01:43.255908 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.255782 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:43.257598 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.257574 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:01:43.258216 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.258187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerStarted","Data":"ba77512b112d76749a1731581995c503ce1acda6335fd8ec8a0d83d234197327"} Jul 10 01:01:43.258309 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.258228 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerStarted","Data":"8454f9b1ab8f1d9cfa8ac40a873addba201590faf9ce5a61f68fe75bf953d871"} Jul 10 01:01:43.258309 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.258243 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerStarted","Data":"6711e79b2d603551d023f8d8b4e5b0d9319d4f3ac2505653d89aed47d6e6796f"} Jul 10 01:01:43.271678 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.271634 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" podStartSLOduration=1.020567461 podStartE2EDuration="6.271617872s" totalImagesPullingTime="5.251050411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:43.270828369 +0000 UTC m=+78.932622723" watchObservedRunningTime="2026-07-10 01:01:43.271617872 +0000 UTC m=+78.933412226" Jul 10 01:01:43.305515 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:01:43.305404 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" podStartSLOduration=1.07559158 podStartE2EDuration="6.305386721s" totalImagesPullingTime="5.229795141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:01:43.304826422 +0000 UTC m=+78.966620778" watchObservedRunningTime="2026-07-10 01:01:43.305386721 +0000 UTC m=+78.967181075" Jul 10 01:02:04.039856 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:02:04.039795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:02:04.040214 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:02:04.039981 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:02:04.040214 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:02:04.040070 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:03:08.040048351 +0000 UTC m=+163.701842688 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:02:07.236619 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:02:07.236589 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k2xjk" Jul 10 01:02:34.661216 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:02:34.661174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:02:34.661710 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:02:34.661303 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 01:02:34.661710 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:02:34.661359 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:04:36.661344436 +0000 UTC m=+252.323138768 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : secret "metrics-daemon-secret" not found Jul 10 01:03:03.193450 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:03:03.193400 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nvklj" podUID="e1e570c0-1149-4943-bffc-5d2f0f8c216d" Jul 10 01:03:03.492727 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:03.492644 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvklj" Jul 10 01:03:04.951465 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:03:04.951419 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:03:08.097101 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:08.097049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:03:08.097581 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:03:08.097200 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 01:03:08.097581 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:03:08.097266 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls podName:e1e570c0-1149-4943-bffc-5d2f0f8c216d nodeName:}" failed. No retries permitted until 2026-07-10 01:05:10.097248965 +0000 UTC m=+285.759043297 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls") pod "dns-default-nvklj" (UID: "e1e570c0-1149-4943-bffc-5d2f0f8c216d") : secret "dns-default-metrics-tls" not found Jul 10 01:03:17.937434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:17.937334 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:03:43.257144 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:43.257073 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" podUID="d95b5a8d-0fc7-46d1-9148-f003b6b25d0f" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Jul 10 01:03:43.612027 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:43.611996 2565 generic.go:356] "Generic (PLEG): container finished" podID="d95b5a8d-0fc7-46d1-9148-f003b6b25d0f" containerID="2885a91de69f5fb6186f4f39c54dc913f7b4f38d5990d151dd76a97975ac47e8" exitCode=1 Jul 10 01:03:43.612198 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:43.612058 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" event={"ID":"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f","Type":"ContainerDied","Data":"2885a91de69f5fb6186f4f39c54dc913f7b4f38d5990d151dd76a97975ac47e8"} Jul 10 01:03:43.612387 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:43.612373 2565 scope.go:117] "RemoveContainer" containerID="2885a91de69f5fb6186f4f39c54dc913f7b4f38d5990d151dd76a97975ac47e8" Jul 10 01:03:44.616119 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:44.616081 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" event={"ID":"d95b5a8d-0fc7-46d1-9148-f003b6b25d0f","Type":"ContainerStarted","Data":"2a8a933c9d297196614dc88d034634e3e6a05f6ce6d76b270c8f87694c8e2ab4"} Jul 10 01:03:44.616509 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:44.616376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:03:44.616986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:03:44.616963 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667b76d967-bcgsp" Jul 10 01:04:30.349312 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.349279 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qpsz6"] Jul 10 01:04:30.351638 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.351618 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.354416 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.354388 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 01:04:30.354583 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.354415 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 01:04:30.354583 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.354432 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 01:04:30.354583 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.354537 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2x8rz\"" Jul 10 01:04:30.354865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.354843 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 01:04:30.362716 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.362692 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qpsz6"] Jul 10 01:04:30.462270 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.462233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fab23a67-fd7e-4676-a278-0bb5eb91987c-crio-socket\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.462270 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.462271 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab23a67-fd7e-4676-a278-0bb5eb91987c-data-volume\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.462270 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.462293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4p7\" (UniqueName: \"kubernetes.io/projected/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-api-access-tf4p7\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.462555 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.462349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.462555 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.462382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563374 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563333 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563449 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563492 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fab23a67-fd7e-4676-a278-0bb5eb91987c-crio-socket\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563492 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:30.563468 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 01:04:30.563492 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab23a67-fd7e-4676-a278-0bb5eb91987c-data-volume\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563502 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4p7\" (UniqueName: \"kubernetes.io/projected/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-api-access-tf4p7\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563632 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:30.563543 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:31.063522967 +0000 UTC m=+246.725317321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : configmap "kube-rbac-proxy" not found Jul 10 01:04:30.563632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563580 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fab23a67-fd7e-4676-a278-0bb5eb91987c-crio-socket\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.563632 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:30.563600 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:30.563764 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:30.563677 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:31.063659786 +0000 UTC m=+246.725454138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:30.563879 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.563859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab23a67-fd7e-4676-a278-0bb5eb91987c-data-volume\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:30.571797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:30.571769 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4p7\" (UniqueName: \"kubernetes.io/projected/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-api-access-tf4p7\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:31.067986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:31.067933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:31.067986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:31.067984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:31.068210 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:31.068078 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 01:04:31.068210 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:31.068111 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:31.068210 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:31.068151 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:32.068134399 +0000 UTC m=+247.729928733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : configmap "kube-rbac-proxy" not found Jul 10 01:04:31.068210 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:31.068167 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:32.068160883 +0000 UTC m=+247.729955217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:32.077338 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:32.077299 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:32.077338 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:32.077338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:32.077869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:32.077439 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 01:04:32.077869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:32.077523 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:34.077501366 +0000 UTC m=+249.739295715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : configmap "kube-rbac-proxy" not found Jul 10 01:04:32.077869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:32.077446 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:32.077869 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:32.077582 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:34.07756978 +0000 UTC m=+249.739364114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:33.140742 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:33.140716 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q9pks_1833f960-875b-4907-90c8-bcdb3c5ffb41/dns-node-resolver/0.log" Jul 10 01:04:33.942865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:33.942834 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5kcqk_894b8144-59f5-4387-b59b-4564ea772cf7/node-ca/0.log" Jul 10 01:04:34.094397 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:34.094363 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:34.094397 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:34.094404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:34.094663 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:34.094469 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 01:04:34.094663 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:34.094546 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:38.094525934 +0000 UTC m=+253.756320266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : configmap "kube-rbac-proxy" not found Jul 10 01:04:34.094663 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:34.094546 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:34.094663 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:34.094609 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:38.094593754 +0000 UTC m=+253.756388089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:35.141429 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:35.141378 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbxdj_6ab167a5-1592-4033-ba8e-d9c05b9d17d1/serve-healthcheck-canary/0.log" Jul 10 01:04:36.711538 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:36.711483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:04:36.712064 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:36.711635 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 01:04:36.712064 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:36.711701 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs podName:cf888257-f4f1-47c2-a54b-eba014c092b6 nodeName:}" failed. No retries permitted until 2026-07-10 01:06:38.711684612 +0000 UTC m=+374.373478946 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs") pod "network-metrics-daemon-rztvp" (UID: "cf888257-f4f1-47c2-a54b-eba014c092b6") : secret "metrics-daemon-secret" not found Jul 10 01:04:38.121686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:38.121638 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:38.121686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:38.121688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:38.122175 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:38.121775 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 01:04:38.122175 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:38.121803 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:38.122175 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:38.121846 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:46.121831085 +0000 UTC m=+261.783625418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : configmap "kube-rbac-proxy" not found Jul 10 01:04:38.122175 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:38.121861 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:04:46.121854881 +0000 UTC m=+261.783649215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:46.181167 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:46.181058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:46.181167 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:46.181111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:46.181695 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:46.181213 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 01:04:46.181695 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:04:46.181267 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls podName:fab23a67-fd7e-4676-a278-0bb5eb91987c nodeName:}" failed. No retries permitted until 2026-07-10 01:05:02.181252755 +0000 UTC m=+277.843047088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpsz6" (UID: "fab23a67-fd7e-4676-a278-0bb5eb91987c") : secret "insights-runtime-extractor-tls" not found Jul 10 01:04:46.181797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:46.181729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fab23a67-fd7e-4676-a278-0bb5eb91987c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:04:52.919729 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.919688 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b46fb8c9d-x4l6d"] Jul 10 01:04:52.921699 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.921677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:52.924525 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.924494 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 01:04:52.924692 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.924673 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 01:04:52.925609 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.925588 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 01:04:52.925609 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.925602 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ghnd6\"" Jul 10 01:04:52.946191 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.946157 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 01:04:52.949557 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:52.949533 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b46fb8c9d-x4l6d"] Jul 10 01:04:53.035740 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-image-registry-private-configuration\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.035740 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-bound-sa-token\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-certificates\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-ca-trust-extracted\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035936 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-installation-pull-secrets\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.035967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-trusted-ca\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.036011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s9cq\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-kube-api-access-7s9cq\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.036066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.036041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-tls\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137125 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9cq\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-kube-api-access-7s9cq\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137125 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-tls\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137232 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-image-registry-private-configuration\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-bound-sa-token\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137290 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-certificates\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137458 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137362 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-ca-trust-extracted\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137458 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-installation-pull-secrets\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137458 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-trusted-ca\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.137863 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.137837 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-ca-trust-extracted\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.138588 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.138551 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-certificates\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.138760 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.138722 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-trusted-ca\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.139880 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.139850 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-registry-tls\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.139880 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.139854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-image-registry-private-configuration\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.140080 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.140057 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-installation-pull-secrets\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.148211 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.148169 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-bound-sa-token\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.148327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.148298 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s9cq\" (UniqueName: \"kubernetes.io/projected/30c4f995-51cf-4b5e-adcc-2819afcfb7b6-kube-api-access-7s9cq\") pod \"image-registry-7b46fb8c9d-x4l6d\" (UID: \"30c4f995-51cf-4b5e-adcc-2819afcfb7b6\") " pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.231047 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.230953 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.361301 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.361254 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b46fb8c9d-x4l6d"] Jul 10 01:04:53.364039 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:04:53.364006 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c4f995_51cf_4b5e_adcc_2819afcfb7b6.slice/crio-256d94be9f0f0f7ecdf2f0c4d04ef600f8d96f7d81ec233c9ed12701491d2b77 WatchSource:0}: Error finding container 256d94be9f0f0f7ecdf2f0c4d04ef600f8d96f7d81ec233c9ed12701491d2b77: Status 404 returned error can't find the container with id 256d94be9f0f0f7ecdf2f0c4d04ef600f8d96f7d81ec233c9ed12701491d2b77 Jul 10 01:04:53.817881 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.817844 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" event={"ID":"30c4f995-51cf-4b5e-adcc-2819afcfb7b6","Type":"ContainerStarted","Data":"cd75f6b7ae99cfb5934b331d327abd5aa752fef7973fa3abcb79d02c3c1943f9"} Jul 10 01:04:53.818078 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.817909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" event={"ID":"30c4f995-51cf-4b5e-adcc-2819afcfb7b6","Type":"ContainerStarted","Data":"256d94be9f0f0f7ecdf2f0c4d04ef600f8d96f7d81ec233c9ed12701491d2b77"} Jul 10 01:04:53.818078 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.817962 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:04:53.837103 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:04:53.837055 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" podStartSLOduration=1.837036525 podStartE2EDuration="1.837036525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:04:53.8364501 +0000 UTC m=+269.498244455" watchObservedRunningTime="2026-07-10 01:04:53.837036525 +0000 UTC m=+269.498830879" Jul 10 01:05:02.204186 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:02.204139 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:05:02.206707 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:02.206681 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fab23a67-fd7e-4676-a278-0bb5eb91987c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpsz6\" (UID: \"fab23a67-fd7e-4676-a278-0bb5eb91987c\") " pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:05:02.460516 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:02.460419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qpsz6" Jul 10 01:05:02.594301 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:02.594267 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qpsz6"] Jul 10 01:05:02.597397 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:05:02.597369 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab23a67_fd7e_4676_a278_0bb5eb91987c.slice/crio-eb718555cffcc96e4cd4f962c9aec965a9444b514faa9b5376b49b2bb75dbe67 WatchSource:0}: Error finding container eb718555cffcc96e4cd4f962c9aec965a9444b514faa9b5376b49b2bb75dbe67: Status 404 returned error can't find the container with id eb718555cffcc96e4cd4f962c9aec965a9444b514faa9b5376b49b2bb75dbe67 Jul 10 01:05:02.844281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:02.844249 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpsz6" event={"ID":"fab23a67-fd7e-4676-a278-0bb5eb91987c","Type":"ContainerStarted","Data":"eb718555cffcc96e4cd4f962c9aec965a9444b514faa9b5376b49b2bb75dbe67"} Jul 10 01:05:06.493405 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:05:06.493357 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nvklj" podUID="e1e570c0-1149-4943-bffc-5d2f0f8c216d" Jul 10 01:05:06.858822 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:06.858792 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:06.858822 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:06.858811 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpsz6" event={"ID":"fab23a67-fd7e-4676-a278-0bb5eb91987c","Type":"ContainerStarted","Data":"48c675f62fbf97613e10244a838584f55de27d9e51502cc39b54193418d24930"} Jul 10 01:05:07.864779 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:07.864742 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpsz6" event={"ID":"fab23a67-fd7e-4676-a278-0bb5eb91987c","Type":"ContainerStarted","Data":"a10b8bcf9c753423b72a72c2a18e180b805e35f0b29cc14de1e6fc7cb9a9acc8"} Jul 10 01:05:09.874104 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:09.874067 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpsz6" event={"ID":"fab23a67-fd7e-4676-a278-0bb5eb91987c","Type":"ContainerStarted","Data":"c0d08c4bdcaf9d46e673d9abdf137bf8e139ec5010ab130bd825438ea0897cca"} Jul 10 01:05:09.893857 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:09.893798 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qpsz6" podStartSLOduration=32.921764074 podStartE2EDuration="39.893780953s" totalImagesPullingTime="6.972016879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:04:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:09.892037766 +0000 UTC m=+285.553832120" watchObservedRunningTime="2026-07-10 01:05:09.893780953 +0000 UTC m=+285.555575308" Jul 10 01:05:10.168134 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.168087 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:10.170597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.170564 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1e570c0-1149-4943-bffc-5d2f0f8c216d-metrics-tls\") pod \"dns-default-nvklj\" (UID: \"e1e570c0-1149-4943-bffc-5d2f0f8c216d\") " pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:10.462279 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.462193 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bmxj4\"" Jul 10 01:05:10.470460 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.470430 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:10.595720 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.595686 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvklj"] Jul 10 01:05:10.598503 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:05:10.598479 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e570c0_1149_4943_bffc_5d2f0f8c216d.slice/crio-2a11f5968691ce8b204ac393bbb13ba221d61de9283ab91ab1a4c7ede6f2e42b WatchSource:0}: Error finding container 2a11f5968691ce8b204ac393bbb13ba221d61de9283ab91ab1a4c7ede6f2e42b: Status 404 returned error can't find the container with id 2a11f5968691ce8b204ac393bbb13ba221d61de9283ab91ab1a4c7ede6f2e42b Jul 10 01:05:10.878651 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:10.878617 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvklj" event={"ID":"e1e570c0-1149-4943-bffc-5d2f0f8c216d","Type":"ContainerStarted","Data":"2a11f5968691ce8b204ac393bbb13ba221d61de9283ab91ab1a4c7ede6f2e42b"} Jul 10 01:05:11.060201 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.060161 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w8ngn"] Jul 10 01:05:11.063345 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.063318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.065936 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.065911 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 01:05:11.065936 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.065922 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 01:05:11.066935 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.066904 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 01:05:11.067053 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.066969 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h87x4\"" Jul 10 01:05:11.067053 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.066982 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 01:05:11.067053 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.066995 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 01:05:11.067191 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.067108 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 01:05:11.177204 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177108 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-sys\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177204 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177204 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz7j8\" (UniqueName: \"kubernetes.io/projected/a9b45868-72c9-4fef-b531-f1913279583a-kube-api-access-cz7j8\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177204 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177258 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-wtmp\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-textfile\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-metrics-client-ca\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.177455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.177378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-root\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.278703 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-wtmp\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.278872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.278872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-textfile\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.278872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-metrics-client-ca\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.278872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-root\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-sys\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:05:11.278935 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278958 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz7j8\" (UniqueName: \"kubernetes.io/projected/a9b45868-72c9-4fef-b531-f1913279583a-kube-api-access-cz7j8\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:05:11.279019 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls podName:a9b45868-72c9-4fef-b531-f1913279583a nodeName:}" failed. No retries permitted until 2026-07-10 01:05:11.778997274 +0000 UTC m=+287.440791620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls") pod "node-exporter-w8ngn" (UID: "a9b45868-72c9-4fef-b531-f1913279583a") : secret "node-exporter-tls" not found Jul 10 01:05:11.279165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.278871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-wtmp\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279502 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.279226 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-root\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279502 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.279276 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9b45868-72c9-4fef-b531-f1913279583a-sys\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.279525 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-metrics-client-ca\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.279574 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.279783 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.279763 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-textfile\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.281826 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.281805 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.288924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.288878 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz7j8\" (UniqueName: \"kubernetes.io/projected/a9b45868-72c9-4fef-b531-f1913279583a-kube-api-access-cz7j8\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.783781 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.783746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.786636 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.786603 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9b45868-72c9-4fef-b531-f1913279583a-node-exporter-tls\") pod \"node-exporter-w8ngn\" (UID: \"a9b45868-72c9-4fef-b531-f1913279583a\") " pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.973120 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:11.973086 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8ngn" Jul 10 01:05:11.983828 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:05:11.983791 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b45868_72c9_4fef_b531_f1913279583a.slice/crio-fcec05a1f708ea8ed5c480048930b7e3ccb9b44151b03c0fa774052438488118 WatchSource:0}: Error finding container fcec05a1f708ea8ed5c480048930b7e3ccb9b44151b03c0fa774052438488118: Status 404 returned error can't find the container with id fcec05a1f708ea8ed5c480048930b7e3ccb9b44151b03c0fa774052438488118 Jul 10 01:05:12.886900 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:12.886861 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvklj" event={"ID":"e1e570c0-1149-4943-bffc-5d2f0f8c216d","Type":"ContainerStarted","Data":"5231a37cb37a555b8f01a5373022266d48b3e737d8e8b0795e0a4f0362024153"} Jul 10 01:05:12.887066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:12.886945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvklj" event={"ID":"e1e570c0-1149-4943-bffc-5d2f0f8c216d","Type":"ContainerStarted","Data":"2cb7963d7aac8e45faca9653ada103f049a4c4da4a4c9ea9a64d60a2821b3cfc"} Jul 10 01:05:12.887066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:12.887006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:12.888178 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:12.888151 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8ngn" event={"ID":"a9b45868-72c9-4fef-b531-f1913279583a","Type":"ContainerStarted","Data":"fcec05a1f708ea8ed5c480048930b7e3ccb9b44151b03c0fa774052438488118"} Jul 10 01:05:12.904181 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:12.904130 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nvklj" podStartSLOduration=251.004333059 podStartE2EDuration="4m12.904111068s" totalImagesPullingTime="1.899778009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:12.90337717 +0000 UTC m=+288.565171524" watchObservedRunningTime="2026-07-10 01:05:12.904111068 +0000 UTC m=+288.565905425" Jul 10 01:05:13.236481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:13.236395 2565 patch_prober.go:28] interesting pod/image-registry-7b46fb8c9d-x4l6d container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 01:05:13.236481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:13.236463 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" podUID="30c4f995-51cf-4b5e-adcc-2819afcfb7b6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 01:05:13.894826 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:13.893654 2565 generic.go:356] "Generic (PLEG): container finished" podID="a9b45868-72c9-4fef-b531-f1913279583a" containerID="262bbbc6cf31bbf960d11d13b09974c84b5dc48514b4a52907fc564e6fc2acf4" exitCode=0 Jul 10 01:05:13.895047 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:13.894841 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8ngn" event={"ID":"a9b45868-72c9-4fef-b531-f1913279583a","Type":"ContainerDied","Data":"262bbbc6cf31bbf960d11d13b09974c84b5dc48514b4a52907fc564e6fc2acf4"} Jul 10 01:05:14.824748 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:14.824715 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b46fb8c9d-x4l6d" Jul 10 01:05:14.898966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:14.898933 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8ngn" event={"ID":"a9b45868-72c9-4fef-b531-f1913279583a","Type":"ContainerStarted","Data":"41a39291373e8be6b5ea1d00ac0075bb27a9e70e290df47d7f6f5675c65206fe"} Jul 10 01:05:14.899127 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:14.898992 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8ngn" event={"ID":"a9b45868-72c9-4fef-b531-f1913279583a","Type":"ContainerStarted","Data":"1a19e578685c5b88f810d497681cc2ac4bf18c1cabf7f89925fce5d352efb9a9"} Jul 10 01:05:14.918871 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:14.918821 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w8ngn" podStartSLOduration=2.438162054 podStartE2EDuration="3.918804646s" totalImagesPullingTime="1.406302657s" totalInitContainerRuntime="74.339935ms" isStatefulPod=true podCreationTimestamp="2026-07-10 01:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:14.917757609 +0000 UTC m=+290.579551966" watchObservedRunningTime="2026-07-10 01:05:14.918804646 +0000 UTC m=+290.580599037" Jul 10 01:05:16.238755 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.238720 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z"] Jul 10 01:05:16.242015 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.241989 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.244772 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.244741 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 01:05:16.244772 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.244769 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 01:05:16.245018 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.245001 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 01:05:16.245082 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.245058 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kwhcc\"" Jul 10 01:05:16.245082 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.245067 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 01:05:16.245314 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.245301 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 01:05:16.249986 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.249962 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 01:05:16.254515 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.254493 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z"] Jul 10 01:05:16.319834 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319789 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.319834 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-metrics-client-ca\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320088 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320088 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320088 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rxz\" (UniqueName: \"kubernetes.io/projected/34b386a4-8a51-4a7b-91b4-0f0640f1783d-kube-api-access-x2rxz\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320088 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.319971 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-serving-certs-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320218 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.320101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-federate-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.320218 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.320135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420657 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420606 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-federate-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420657 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420660 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-metrics-client-ca\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420755 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rxz\" (UniqueName: \"kubernetes.io/projected/34b386a4-8a51-4a7b-91b4-0f0640f1783d-kube-api-access-x2rxz\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.420828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.420790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-serving-certs-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.421741 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.421677 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-serving-certs-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.421867 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.421759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.422116 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.422088 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34b386a4-8a51-4a7b-91b4-0f0640f1783d-metrics-client-ca\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.423481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.423451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.423591 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.423569 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-secret-telemeter-client\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.423765 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.423750 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-telemeter-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.423797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.423765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34b386a4-8a51-4a7b-91b4-0f0640f1783d-federate-client-tls\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.431585 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.431557 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rxz\" (UniqueName: \"kubernetes.io/projected/34b386a4-8a51-4a7b-91b4-0f0640f1783d-kube-api-access-x2rxz\") pod \"telemeter-client-7d5bb68b45-g5n2z\" (UID: \"34b386a4-8a51-4a7b-91b4-0f0640f1783d\") " pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.551960 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.551867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" Jul 10 01:05:16.689198 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.689069 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z"] Jul 10 01:05:16.693566 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:05:16.693537 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b386a4_8a51_4a7b_91b4_0f0640f1783d.slice/crio-db7ef70255937c29b03ed609c7e520cbd2df47381892ab7fa4306ce8cdfeeb4f WatchSource:0}: Error finding container db7ef70255937c29b03ed609c7e520cbd2df47381892ab7fa4306ce8cdfeeb4f: Status 404 returned error can't find the container with id db7ef70255937c29b03ed609c7e520cbd2df47381892ab7fa4306ce8cdfeeb4f Jul 10 01:05:16.905782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:16.905741 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" event={"ID":"34b386a4-8a51-4a7b-91b4-0f0640f1783d","Type":"ContainerStarted","Data":"db7ef70255937c29b03ed609c7e520cbd2df47381892ab7fa4306ce8cdfeeb4f"} Jul 10 01:05:19.919806 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:19.919762 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" event={"ID":"34b386a4-8a51-4a7b-91b4-0f0640f1783d","Type":"ContainerStarted","Data":"544e618437524752aa0c16790d1dd2f7f7282ed5ffba22052e998d47c4ced271"} Jul 10 01:05:20.925063 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:20.925030 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" event={"ID":"34b386a4-8a51-4a7b-91b4-0f0640f1783d","Type":"ContainerStarted","Data":"136537bf5809188e8f91a09e83e8d3e2cecd62f5f67660ed7c8aa2cff5098dd9"} Jul 10 01:05:20.925435 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:20.925084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" event={"ID":"34b386a4-8a51-4a7b-91b4-0f0640f1783d","Type":"ContainerStarted","Data":"2a3df5169cdde95474f9d7f513c22f669a52c659a7965e29f479db5ee6ee6ac9"} Jul 10 01:05:20.938091 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:05:20.938049 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rztvp" podUID="cf888257-f4f1-47c2-a54b-eba014c092b6" Jul 10 01:05:20.947807 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:20.947759 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d5bb68b45-g5n2z" podStartSLOduration=1.5950374269999998 podStartE2EDuration="4.947742587s" totalImagesPullingTime="3.35270516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:05:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:20.945654026 +0000 UTC m=+296.607448383" watchObservedRunningTime="2026-07-10 01:05:20.947742587 +0000 UTC m=+296.609536941" Jul 10 01:05:22.896805 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:22.896772 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nvklj" Jul 10 01:05:24.805057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:24.805024 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:05:24.805561 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:24.805369 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:05:24.809224 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:24.809203 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 01:05:35.937179 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:05:35.937141 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:06:07.675220 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:07.675179 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" podUID="16dde9a7-f045-4df3-97b3-6b1f1259c7db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 01:06:17.675257 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:17.675163 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" podUID="16dde9a7-f045-4df3-97b3-6b1f1259c7db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 01:06:27.675093 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:27.675054 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" podUID="16dde9a7-f045-4df3-97b3-6b1f1259c7db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 01:06:27.675485 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:27.675123 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" Jul 10 01:06:27.675613 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:27.675586 2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ba77512b112d76749a1731581995c503ce1acda6335fd8ec8a0d83d234197327"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 10 01:06:27.675652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:27.675634 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" podUID="16dde9a7-f045-4df3-97b3-6b1f1259c7db" containerName="service-proxy" containerID="cri-o://ba77512b112d76749a1731581995c503ce1acda6335fd8ec8a0d83d234197327" gracePeriod=30 Jul 10 01:06:28.146367 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:28.146334 2565 generic.go:356] "Generic (PLEG): container finished" podID="16dde9a7-f045-4df3-97b3-6b1f1259c7db" containerID="ba77512b112d76749a1731581995c503ce1acda6335fd8ec8a0d83d234197327" exitCode=2 Jul 10 01:06:28.146539 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:28.146395 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerDied","Data":"ba77512b112d76749a1731581995c503ce1acda6335fd8ec8a0d83d234197327"} Jul 10 01:06:28.146539 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:28.146430 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b9877b597-n6d6d" event={"ID":"16dde9a7-f045-4df3-97b3-6b1f1259c7db","Type":"ContainerStarted","Data":"85666a0aa6796b7ba3234af0ae7201a8944c4c27eca0741bc693d71939dd8e29"} Jul 10 01:06:38.747200 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:38.747154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:06:38.749592 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:38.749561 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf888257-f4f1-47c2-a54b-eba014c092b6-metrics-certs\") pod \"network-metrics-daemon-rztvp\" (UID: \"cf888257-f4f1-47c2-a54b-eba014c092b6\") " pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:06:38.940127 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:38.940099 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjnvz\"" Jul 10 01:06:38.948330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:38.948307 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rztvp" Jul 10 01:06:39.097230 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:39.097192 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rztvp"] Jul 10 01:06:39.100178 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:06:39.100155 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf888257_f4f1_47c2_a54b_eba014c092b6.slice/crio-52e31ca19d5260b040a754207533a520e9f128e4a031db5fac016e9fa7a358c6 WatchSource:0}: Error finding container 52e31ca19d5260b040a754207533a520e9f128e4a031db5fac016e9fa7a358c6: Status 404 returned error can't find the container with id 52e31ca19d5260b040a754207533a520e9f128e4a031db5fac016e9fa7a358c6 Jul 10 01:06:39.102150 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:39.102133 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 01:06:39.185868 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:39.185833 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rztvp" event={"ID":"cf888257-f4f1-47c2-a54b-eba014c092b6","Type":"ContainerStarted","Data":"52e31ca19d5260b040a754207533a520e9f128e4a031db5fac016e9fa7a358c6"} Jul 10 01:06:41.194549 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:41.194504 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rztvp" event={"ID":"cf888257-f4f1-47c2-a54b-eba014c092b6","Type":"ContainerStarted","Data":"19ed6c3cd23c8c68d98c18ab75d0d8cc4743d346608335dd0c9b9eaa761e58cc"} Jul 10 01:06:41.195095 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:41.194612 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rztvp" event={"ID":"cf888257-f4f1-47c2-a54b-eba014c092b6","Type":"ContainerStarted","Data":"5acb9bedc641712fc9b527faf231e2fd00e706a05365911b9cf218b7b5c61850"} Jul 10 01:06:41.210574 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:06:41.210513 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rztvp" podStartSLOduration=375.092189203 podStartE2EDuration="6m16.210497288s" totalImagesPullingTime="1.118308085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:06:41.20906468 +0000 UTC m=+376.870859034" watchObservedRunningTime="2026-07-10 01:06:41.210497288 +0000 UTC m=+376.872291643" Jul 10 01:07:08.327184 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.327151 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk"] Jul 10 01:07:08.329526 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.329508 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.331984 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.331949 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 01:07:08.332115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.332013 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 01:07:08.332115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.332030 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ld2t7\"" Jul 10 01:07:08.338537 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.338511 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk"] Jul 10 01:07:08.467205 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.467170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.467395 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.467242 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.467395 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.467289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpjrm\" (UniqueName: \"kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.568602 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.568562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.568602 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.568600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpjrm\" (UniqueName: \"kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.568797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.568630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.569071 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.569048 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.569122 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.569056 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.577556 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.577484 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpjrm\" (UniqueName: \"kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.639169 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.639125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:08.771558 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:08.771527 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk"] Jul 10 01:07:08.774404 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:07:08.774371 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5252cd92_023f_40c9_a443_bf0838b0dec0.slice/crio-d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8 WatchSource:0}: Error finding container d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8: Status 404 returned error can't find the container with id d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8 Jul 10 01:07:09.290840 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:09.290800 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" event={"ID":"5252cd92-023f-40c9-a443-bf0838b0dec0","Type":"ContainerStarted","Data":"d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8"} Jul 10 01:07:17.322415 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:17.322379 2565 generic.go:356] "Generic (PLEG): container finished" podID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerID="7bce2d99d69094a7672ac826e0ed392fc0ab7f0cd5cbf529707ccfa677aeb3f3" exitCode=0 Jul 10 01:07:17.322925 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:17.322470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" event={"ID":"5252cd92-023f-40c9-a443-bf0838b0dec0","Type":"ContainerDied","Data":"7bce2d99d69094a7672ac826e0ed392fc0ab7f0cd5cbf529707ccfa677aeb3f3"} Jul 10 01:07:20.335913 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:20.335858 2565 generic.go:356] "Generic (PLEG): container finished" podID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerID="0446269b1640f82fecd01f1414dd1c7148e05bd68e0e3a473df33a8b5ab3bdbf" exitCode=0 Jul 10 01:07:20.336264 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:20.335932 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" event={"ID":"5252cd92-023f-40c9-a443-bf0838b0dec0","Type":"ContainerDied","Data":"0446269b1640f82fecd01f1414dd1c7148e05bd68e0e3a473df33a8b5ab3bdbf"} Jul 10 01:07:29.372346 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:29.372301 2565 generic.go:356] "Generic (PLEG): container finished" podID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerID="3fcf4a1bdfa161beb3e93635f2a3f56619da9dec4dbd0b0030e95553158bc60a" exitCode=0 Jul 10 01:07:29.372727 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:29.372399 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" event={"ID":"5252cd92-023f-40c9-a443-bf0838b0dec0","Type":"ContainerDied","Data":"3fcf4a1bdfa161beb3e93635f2a3f56619da9dec4dbd0b0030e95553158bc60a"} Jul 10 01:07:30.552832 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.552795 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:30.642493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.642399 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util\") pod \"5252cd92-023f-40c9-a443-bf0838b0dec0\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " Jul 10 01:07:30.642493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.642440 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle\") pod \"5252cd92-023f-40c9-a443-bf0838b0dec0\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " Jul 10 01:07:30.642493 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.642463 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpjrm\" (UniqueName: \"kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm\") pod \"5252cd92-023f-40c9-a443-bf0838b0dec0\" (UID: \"5252cd92-023f-40c9-a443-bf0838b0dec0\") " Jul 10 01:07:30.643099 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.643072 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle" (OuterVolumeSpecName: "bundle") pod "5252cd92-023f-40c9-a443-bf0838b0dec0" (UID: "5252cd92-023f-40c9-a443-bf0838b0dec0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:07:30.644869 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.644845 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm" (OuterVolumeSpecName: "kube-api-access-jpjrm") pod "5252cd92-023f-40c9-a443-bf0838b0dec0" (UID: "5252cd92-023f-40c9-a443-bf0838b0dec0"). InnerVolumeSpecName "kube-api-access-jpjrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:07:30.646862 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.646828 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util" (OuterVolumeSpecName: "util") pod "5252cd92-023f-40c9-a443-bf0838b0dec0" (UID: "5252cd92-023f-40c9-a443-bf0838b0dec0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:07:30.743155 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.743112 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:30.743155 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.743145 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5252cd92-023f-40c9-a443-bf0838b0dec0-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:30.743155 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:30.743158 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpjrm\" (UniqueName: \"kubernetes.io/projected/5252cd92-023f-40c9-a443-bf0838b0dec0-kube-api-access-jpjrm\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:31.381527 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:31.381490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" event={"ID":"5252cd92-023f-40c9-a443-bf0838b0dec0","Type":"ContainerDied","Data":"d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8"} Jul 10 01:07:31.381690 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:31.381548 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d966662efa7e3c1ffe2c3b553279a7cbf397e5336e0e2364de561a2122ceb9c8" Jul 10 01:07:31.381690 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:31.381507 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9trk" Jul 10 01:07:40.329466 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329435 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw"] Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329768 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="pull" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329782 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="pull" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329794 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="util" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329799 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="util" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329805 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="extract" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329810 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="extract" Jul 10 01:07:40.329916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.329856 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5252cd92-023f-40c9-a443-bf0838b0dec0" containerName="extract" Jul 10 01:07:40.332783 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.332765 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.336173 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.336145 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 01:07:40.336333 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.336195 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ld2t7\"" Jul 10 01:07:40.337048 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.337030 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 01:07:40.368469 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.368434 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw"] Jul 10 01:07:40.409485 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.409445 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.409684 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.409497 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.409684 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.409534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqjl\" (UniqueName: \"kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.510054 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.510013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.510236 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.510069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.510236 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.510108 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqjl\" (UniqueName: \"kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.510433 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.510406 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.510481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.510433 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.518943 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.518883 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqjl\" (UniqueName: \"kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.641748 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.641702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:40.773782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:40.773663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw"] Jul 10 01:07:40.776607 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:07:40.776581 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb916e45a_a0cd_4d71_b4e9_de8d4c6de26f.slice/crio-973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3 WatchSource:0}: Error finding container 973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3: Status 404 returned error can't find the container with id 973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3 Jul 10 01:07:41.417417 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:41.417378 2565 generic.go:356] "Generic (PLEG): container finished" podID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerID="9f9a43d625a17643a06eb9dc7188cdd7168b4312f917ba216a594ff6d9af69b6" exitCode=0 Jul 10 01:07:41.417810 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:41.417424 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" event={"ID":"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f","Type":"ContainerDied","Data":"9f9a43d625a17643a06eb9dc7188cdd7168b4312f917ba216a594ff6d9af69b6"} Jul 10 01:07:41.417810 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:41.417475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" event={"ID":"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f","Type":"ContainerStarted","Data":"973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3"} Jul 10 01:07:46.441032 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:46.440992 2565 generic.go:356] "Generic (PLEG): container finished" podID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerID="0cfa24aaf3e8639b18895d520de1549f246236a306b92ed61f7a9f7910c7113c" exitCode=0 Jul 10 01:07:46.441518 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:46.441060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" event={"ID":"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f","Type":"ContainerDied","Data":"0cfa24aaf3e8639b18895d520de1549f246236a306b92ed61f7a9f7910c7113c"} Jul 10 01:07:47.446787 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:47.446751 2565 generic.go:356] "Generic (PLEG): container finished" podID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerID="d0b6c32253ff62c084b12bfe8aa5d49aa3a4d7526d79c573575cd6ac9dcd69c7" exitCode=0 Jul 10 01:07:47.447198 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:47.446810 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" event={"ID":"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f","Type":"ContainerDied","Data":"d0b6c32253ff62c084b12bfe8aa5d49aa3a4d7526d79c573575cd6ac9dcd69c7"} Jul 10 01:07:48.574440 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.574408 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:48.670582 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.670538 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util\") pod \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " Jul 10 01:07:48.670582 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.670585 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle\") pod \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " Jul 10 01:07:48.670850 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.670604 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgqjl\" (UniqueName: \"kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl\") pod \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\" (UID: \"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f\") " Jul 10 01:07:48.671046 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.671021 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle" (OuterVolumeSpecName: "bundle") pod "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" (UID: "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:07:48.672935 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.672884 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl" (OuterVolumeSpecName: "kube-api-access-bgqjl") pod "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" (UID: "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f"). InnerVolumeSpecName "kube-api-access-bgqjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:07:48.675744 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.675700 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util" (OuterVolumeSpecName: "util") pod "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" (UID: "b916e45a-a0cd-4d71-b4e9-de8d4c6de26f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:07:48.771884 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.771785 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:48.771884 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.771822 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:48.771884 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:48.771834 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgqjl\" (UniqueName: \"kubernetes.io/projected/b916e45a-a0cd-4d71-b4e9-de8d4c6de26f-kube-api-access-bgqjl\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:07:49.456080 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:49.456044 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" event={"ID":"b916e45a-a0cd-4d71-b4e9-de8d4c6de26f","Type":"ContainerDied","Data":"973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3"} Jul 10 01:07:49.456249 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:49.456098 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjw5kw" Jul 10 01:07:49.456249 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:07:49.456101 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="973cf9db4b5018e6ca17ac96fbe7d7e6a4439faf60373fb3c46eb50e823652e3" Jul 10 01:08:00.037676 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037636 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb"] Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037916 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="util" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037929 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="util" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037944 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="extract" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037954 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="extract" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037964 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="pull" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.037969 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="pull" Jul 10 01:08:00.038079 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.038010 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b916e45a-a0cd-4d71-b4e9-de8d4c6de26f" containerName="extract" Jul 10 01:08:00.049539 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.049509 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb"] Jul 10 01:08:00.049717 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.049627 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.052317 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.052291 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ld2t7\"" Jul 10 01:08:00.052481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.052339 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 01:08:00.052481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.052343 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 01:08:00.154797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.154758 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.155000 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.154813 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2jmj\" (UniqueName: \"kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.155000 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.154832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.256281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.256238 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.256423 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.256312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2jmj\" (UniqueName: \"kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.256423 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.256342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.256699 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.256677 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.256761 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.256694 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.264470 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.264442 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2jmj\" (UniqueName: \"kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.360641 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.360610 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:00.512454 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:00.512426 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb"] Jul 10 01:08:00.515103 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:00.515073 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2a544a_432d_46ca_823d_5b06d7df2fd1.slice/crio-e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08 WatchSource:0}: Error finding container e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08: Status 404 returned error can't find the container with id e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08 Jul 10 01:08:01.500860 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:01.500818 2565 generic.go:356] "Generic (PLEG): container finished" podID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerID="42d7ca735ffc5e7982f51a57aca6c9f76dee6c20138b3814a053b439adf0be78" exitCode=0 Jul 10 01:08:01.501247 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:01.500885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" event={"ID":"2c2a544a-432d-46ca-823d-5b06d7df2fd1","Type":"ContainerDied","Data":"42d7ca735ffc5e7982f51a57aca6c9f76dee6c20138b3814a053b439adf0be78"} Jul 10 01:08:01.501247 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:01.500995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" event={"ID":"2c2a544a-432d-46ca-823d-5b06d7df2fd1","Type":"ContainerStarted","Data":"e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08"} Jul 10 01:08:03.509661 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:03.509622 2565 generic.go:356] "Generic (PLEG): container finished" podID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerID="e445f6c79287855b43020b5522001c0b5a9759722dc8dc7eb2402c2b2964f670" exitCode=0 Jul 10 01:08:03.510157 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:03.509670 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" event={"ID":"2c2a544a-432d-46ca-823d-5b06d7df2fd1","Type":"ContainerDied","Data":"e445f6c79287855b43020b5522001c0b5a9759722dc8dc7eb2402c2b2964f670"} Jul 10 01:08:04.516653 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:04.516609 2565 generic.go:356] "Generic (PLEG): container finished" podID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerID="87bd07c642617c4b9c7f68a5841c0728c4869d21bb9a08cf4d441d1bb1c8b1c7" exitCode=0 Jul 10 01:08:04.517059 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:04.516686 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" event={"ID":"2c2a544a-432d-46ca-823d-5b06d7df2fd1","Type":"ContainerDied","Data":"87bd07c642617c4b9c7f68a5841c0728c4869d21bb9a08cf4d441d1bb1c8b1c7"} Jul 10 01:08:05.645339 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.645308 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:05.697945 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.697911 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util\") pod \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " Jul 10 01:08:05.697945 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.697952 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2jmj\" (UniqueName: \"kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj\") pod \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " Jul 10 01:08:05.698194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.697978 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle\") pod \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\" (UID: \"2c2a544a-432d-46ca-823d-5b06d7df2fd1\") " Jul 10 01:08:05.698713 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.698684 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle" (OuterVolumeSpecName: "bundle") pod "2c2a544a-432d-46ca-823d-5b06d7df2fd1" (UID: "2c2a544a-432d-46ca-823d-5b06d7df2fd1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:08:05.700249 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.700225 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj" (OuterVolumeSpecName: "kube-api-access-l2jmj") pod "2c2a544a-432d-46ca-823d-5b06d7df2fd1" (UID: "2c2a544a-432d-46ca-823d-5b06d7df2fd1"). InnerVolumeSpecName "kube-api-access-l2jmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:08:05.704309 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.704283 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util" (OuterVolumeSpecName: "util") pod "2c2a544a-432d-46ca-823d-5b06d7df2fd1" (UID: "2c2a544a-432d-46ca-823d-5b06d7df2fd1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:08:05.798677 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.798584 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:05.798677 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.798618 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2jmj\" (UniqueName: \"kubernetes.io/projected/2c2a544a-432d-46ca-823d-5b06d7df2fd1-kube-api-access-l2jmj\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:05.798677 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:05.798629 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2a544a-432d-46ca-823d-5b06d7df2fd1-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:06.526113 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:06.526078 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" event={"ID":"2c2a544a-432d-46ca-823d-5b06d7df2fd1","Type":"ContainerDied","Data":"e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08"} Jul 10 01:08:06.526277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:06.526130 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e21056b9106e92382d6ced5fd285df740afb121aaa6d96a6864e4d3d240f4a08" Jul 10 01:08:06.526277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:06.526086 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsxddb" Jul 10 01:08:16.865359 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865323 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq"] Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865568 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="util" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865578 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="util" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865586 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="pull" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865591 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="pull" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865600 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="extract" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865605 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="extract" Jul 10 01:08:16.865773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.865646 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2a544a-432d-46ca-823d-5b06d7df2fd1" containerName="extract" Jul 10 01:08:16.867516 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.867498 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:16.870231 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.870205 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 01:08:16.870377 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.870244 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 01:08:16.870377 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.870254 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 01:08:16.870637 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.870623 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 01:08:16.870704 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.870689 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fvs8j\"" Jul 10 01:08:16.885183 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.885152 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq"] Jul 10 01:08:16.980916 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.980841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:16.981129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.980943 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-tmp\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:16.981129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.980990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:16.981129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.981048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gftc\" (UniqueName: \"kubernetes.io/projected/c99fb414-fef1-4d3b-a091-4159cde2fc00-kube-api-access-4gftc\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:16.981129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:16.981121 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-manifests\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.081780 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.081740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-manifests\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.081966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.081793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.081966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.081832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-tmp\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.081966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.081863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.081966 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.081935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gftc\" (UniqueName: \"kubernetes.io/projected/c99fb414-fef1-4d3b-a091-4159cde2fc00-kube-api-access-4gftc\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.082270 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.082244 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-manifests\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.084348 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.084322 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c99fb414-fef1-4d3b-a091-4159cde2fc00-tmp\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.084491 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.084468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.084557 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.084539 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c99fb414-fef1-4d3b-a091-4159cde2fc00-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.100988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.100959 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gftc\" (UniqueName: \"kubernetes.io/projected/c99fb414-fef1-4d3b-a091-4159cde2fc00-kube-api-access-4gftc\") pod \"opendatahub-operator-controller-manager-9c74cbdcf-tstbq\" (UID: \"c99fb414-fef1-4d3b-a091-4159cde2fc00\") " pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.178353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.178258 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:17.323073 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.323037 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq"] Jul 10 01:08:17.326334 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:17.326306 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99fb414_fef1_4d3b_a091_4159cde2fc00.slice/crio-77555ca19c5bba3f0e95bdf9415a6b566b65e08cbdfc757871a27a49d665c6d9 WatchSource:0}: Error finding container 77555ca19c5bba3f0e95bdf9415a6b566b65e08cbdfc757871a27a49d665c6d9: Status 404 returned error can't find the container with id 77555ca19c5bba3f0e95bdf9415a6b566b65e08cbdfc757871a27a49d665c6d9 Jul 10 01:08:17.564014 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:17.563923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" event={"ID":"c99fb414-fef1-4d3b-a091-4159cde2fc00","Type":"ContainerStarted","Data":"77555ca19c5bba3f0e95bdf9415a6b566b65e08cbdfc757871a27a49d665c6d9"} Jul 10 01:08:20.577397 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:20.577305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" event={"ID":"c99fb414-fef1-4d3b-a091-4159cde2fc00","Type":"ContainerStarted","Data":"d52831e3926d8520f3b84bb72a17fb6a1a2c2016d0b6c890c9f7cbd33a75936a"} Jul 10 01:08:21.582761 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:21.582662 2565 generic.go:356] "Generic (PLEG): container finished" podID="c99fb414-fef1-4d3b-a091-4159cde2fc00" containerID="d52831e3926d8520f3b84bb72a17fb6a1a2c2016d0b6c890c9f7cbd33a75936a" exitCode=0 Jul 10 01:08:21.583146 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:21.582764 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" event={"ID":"c99fb414-fef1-4d3b-a091-4159cde2fc00","Type":"ContainerDied","Data":"d52831e3926d8520f3b84bb72a17fb6a1a2c2016d0b6c890c9f7cbd33a75936a"} Jul 10 01:08:22.590186 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:22.590144 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" event={"ID":"c99fb414-fef1-4d3b-a091-4159cde2fc00","Type":"ContainerStarted","Data":"de4e43b5a2d7c3c99ab003d0d2803e6b70d178546e3f399a82f76fb84f650182"} Jul 10 01:08:22.590585 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:22.590262 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:22.614867 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:22.614794 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" podStartSLOduration=2.245503411 podStartE2EDuration="6.614770187s" totalImagesPullingTime="3.365120635s" totalInitContainerRuntime="1.004146141s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:22.611594359 +0000 UTC m=+478.273388714" watchObservedRunningTime="2026-07-10 01:08:22.614770187 +0000 UTC m=+478.276564542" Jul 10 01:08:33.596415 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:33.596380 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9c74cbdcf-tstbq" Jul 10 01:08:35.990839 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:35.990805 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl"] Jul 10 01:08:36.000526 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.000498 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.003744 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.003716 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 01:08:36.003925 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.003745 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 01:08:36.004002 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.003956 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 01:08:36.004109 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.004083 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl"] Jul 10 01:08:36.004762 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.004743 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2x9vw\"" Jul 10 01:08:36.004831 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.004746 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 01:08:36.130832 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.130798 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tmp\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.131023 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.130841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tls-certs\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.131023 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.130863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhsgj\" (UniqueName: \"kubernetes.io/projected/b62948b9-ed2a-488c-80e4-31aa5d73bddb-kube-api-access-dhsgj\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.184232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.184201 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqzx2"] Jul 10 01:08:36.187448 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.187423 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.190438 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.190403 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 01:08:36.190594 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.190444 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xjzrc\"" Jul 10 01:08:36.202716 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.202684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqzx2"] Jul 10 01:08:36.234240 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.234200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tmp\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.234495 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.234477 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tls-certs\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.234610 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.234594 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhsgj\" (UniqueName: \"kubernetes.io/projected/b62948b9-ed2a-488c-80e4-31aa5d73bddb-kube-api-access-dhsgj\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.237776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.237743 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tmp\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.239277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.239249 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62948b9-ed2a-488c-80e4-31aa5d73bddb-tls-certs\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.243933 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.243846 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhsgj\" (UniqueName: \"kubernetes.io/projected/b62948b9-ed2a-488c-80e4-31aa5d73bddb-kube-api-access-dhsgj\") pod \"kube-auth-proxy-5cd866567d-p5rjl\" (UID: \"b62948b9-ed2a-488c-80e4-31aa5d73bddb\") " pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.312179 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.312145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" Jul 10 01:08:36.335667 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.335637 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56pq\" (UniqueName: \"kubernetes.io/projected/6d714dfa-a9d0-4f33-9916-03246b7769fe-kube-api-access-h56pq\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.335831 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.335714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d714dfa-a9d0-4f33-9916-03246b7769fe-cert\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.436565 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.436516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d714dfa-a9d0-4f33-9916-03246b7769fe-cert\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.436776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.436607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56pq\" (UniqueName: \"kubernetes.io/projected/6d714dfa-a9d0-4f33-9916-03246b7769fe-kube-api-access-h56pq\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.439514 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.439484 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d714dfa-a9d0-4f33-9916-03246b7769fe-cert\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.448710 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.448680 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56pq\" (UniqueName: \"kubernetes.io/projected/6d714dfa-a9d0-4f33-9916-03246b7769fe-kube-api-access-h56pq\") pod \"odh-model-controller-7998c67dc4-hqzx2\" (UID: \"6d714dfa-a9d0-4f33-9916-03246b7769fe\") " pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.468558 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.468535 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl"] Jul 10 01:08:36.470743 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:36.470717 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62948b9_ed2a_488c_80e4_31aa5d73bddb.slice/crio-74bbf1dfc68e5a21dbf4d448220a1b6f0e9b3362fb68215666498eeaebb79d70 WatchSource:0}: Error finding container 74bbf1dfc68e5a21dbf4d448220a1b6f0e9b3362fb68215666498eeaebb79d70: Status 404 returned error can't find the container with id 74bbf1dfc68e5a21dbf4d448220a1b6f0e9b3362fb68215666498eeaebb79d70 Jul 10 01:08:36.498765 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.498660 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:36.643193 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.643037 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hqzx2"] Jul 10 01:08:36.644576 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:36.644539 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" event={"ID":"b62948b9-ed2a-488c-80e4-31aa5d73bddb","Type":"ContainerStarted","Data":"74bbf1dfc68e5a21dbf4d448220a1b6f0e9b3362fb68215666498eeaebb79d70"} Jul 10 01:08:36.645328 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:36.645305 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d714dfa_a9d0_4f33_9916_03246b7769fe.slice/crio-21813ca8680f655a2d02f5d0180f8eb8c65aa0c89ce067193c9a0c6f54068809 WatchSource:0}: Error finding container 21813ca8680f655a2d02f5d0180f8eb8c65aa0c89ce067193c9a0c6f54068809: Status 404 returned error can't find the container with id 21813ca8680f655a2d02f5d0180f8eb8c65aa0c89ce067193c9a0c6f54068809 Jul 10 01:08:37.068527 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.068492 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b96gk"] Jul 10 01:08:37.072783 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.072763 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.075155 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.075124 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 01:08:37.083387 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.083346 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b96gk"] Jul 10 01:08:37.244429 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.244395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv8bf\" (UniqueName: \"kubernetes.io/projected/8486740b-c1e4-4264-ba54-ee7b7cba2806-kube-api-access-pv8bf\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.244429 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.244433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8486740b-c1e4-4264-ba54-ee7b7cba2806-tls-certs\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.261010 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.260976 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:37.264021 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.264000 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.266872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.266453 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 01:08:37.266872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.266595 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 01:08:37.266872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.266669 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 01:08:37.266872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.266730 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4hmk9\"" Jul 10 01:08:37.266872 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.266821 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 01:08:37.282385 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.282351 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:37.345980 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.345871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv8bf\" (UniqueName: \"kubernetes.io/projected/8486740b-c1e4-4264-ba54-ee7b7cba2806-kube-api-access-pv8bf\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.346187 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.346011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8486740b-c1e4-4264-ba54-ee7b7cba2806-tls-certs\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.349009 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.348977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8486740b-c1e4-4264-ba54-ee7b7cba2806-tls-certs\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.354360 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.354329 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv8bf\" (UniqueName: \"kubernetes.io/projected/8486740b-c1e4-4264-ba54-ee7b7cba2806-kube-api-access-pv8bf\") pod \"model-serving-api-54d89bd5fc-b96gk\" (UID: \"8486740b-c1e4-4264-ba54-ee7b7cba2806\") " pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.385061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.385023 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:37.446792 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.446792 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982qg\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.446792 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446745 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.447115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446826 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.447115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.447115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.447115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.446964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548137 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548170 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982qg\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548281 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.548330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.548315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.549315 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.549230 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.551162 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.551129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.551275 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.551231 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.551275 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.551257 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.551656 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.551631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.558327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.558283 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b96gk"] Jul 10 01:08:37.559762 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.559733 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.560656 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:37.560623 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8486740b_c1e4_4264_ba54_ee7b7cba2806.slice/crio-56ff74403602a8a221a6e1fb4ddb4cbfcfd721a0d2426b0f9aa89220fb3dcc28 WatchSource:0}: Error finding container 56ff74403602a8a221a6e1fb4ddb4cbfcfd721a0d2426b0f9aa89220fb3dcc28: Status 404 returned error can't find the container with id 56ff74403602a8a221a6e1fb4ddb4cbfcfd721a0d2426b0f9aa89220fb3dcc28 Jul 10 01:08:37.561054 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.561026 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-982qg\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg\") pod \"istiod-openshift-gateway-7574459656-d4r48\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.577057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.577018 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:37.655317 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.655268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" event={"ID":"8486740b-c1e4-4264-ba54-ee7b7cba2806","Type":"ContainerStarted","Data":"56ff74403602a8a221a6e1fb4ddb4cbfcfd721a0d2426b0f9aa89220fb3dcc28"} Jul 10 01:08:37.656874 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.656827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" event={"ID":"6d714dfa-a9d0-4f33-9916-03246b7769fe","Type":"ContainerStarted","Data":"21813ca8680f655a2d02f5d0180f8eb8c65aa0c89ce067193c9a0c6f54068809"} Jul 10 01:08:37.748215 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:37.748039 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:37.751166 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:37.751131 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9dac652_2c60_40c8_a115_36f9c378a36b.slice/crio-f23df7e5a36708f05e13da013325b22ecff4a7c650f321066ea9a816b7c4133a WatchSource:0}: Error finding container f23df7e5a36708f05e13da013325b22ecff4a7c650f321066ea9a816b7c4133a: Status 404 returned error can't find the container with id f23df7e5a36708f05e13da013325b22ecff4a7c650f321066ea9a816b7c4133a Jul 10 01:08:38.666071 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:38.666000 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" event={"ID":"e9dac652-2c60-40c8-a115-36f9c378a36b","Type":"ContainerStarted","Data":"f23df7e5a36708f05e13da013325b22ecff4a7c650f321066ea9a816b7c4133a"} Jul 10 01:08:42.616661 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:42.616613 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 01:08:42.617209 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:42.616708 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 01:08:43.703556 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.703523 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" event={"ID":"6d714dfa-a9d0-4f33-9916-03246b7769fe","Type":"ContainerStarted","Data":"39e799204ea3cd6e3bf272d48d4d758402188390e591990d6e22d46bc7fd6d0c"} Jul 10 01:08:43.704081 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.703625 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:43.705611 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.705576 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" event={"ID":"e9dac652-2c60-40c8-a115-36f9c378a36b","Type":"ContainerStarted","Data":"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549"} Jul 10 01:08:43.705797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.705640 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:43.707449 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.707417 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" event={"ID":"b62948b9-ed2a-488c-80e4-31aa5d73bddb","Type":"ContainerStarted","Data":"76ba9b52f57605a27ad090209d365cdb5ec1132f22520ee841f15f0f8ffd77dd"} Jul 10 01:08:43.708090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.708053 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-d4r48 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 01:08:43.708228 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.708118 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 01:08:43.709246 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.709224 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" event={"ID":"8486740b-c1e4-4264-ba54-ee7b7cba2806","Type":"ContainerStarted","Data":"868d690ca46a4d581e4dcd16bfe943468c0075295b6d8489a7c2f1adf276f392"} Jul 10 01:08:43.709382 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.709365 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:08:43.725605 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.725542 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" podStartSLOduration=1.755797555 podStartE2EDuration="7.725524841s" totalImagesPullingTime="5.969727286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:43.724265744 +0000 UTC m=+499.386060101" watchObservedRunningTime="2026-07-10 01:08:43.725524841 +0000 UTC m=+499.387319196" Jul 10 01:08:43.746135 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.746069 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" podStartSLOduration=1.883975758 podStartE2EDuration="6.746052656s" totalImagesPullingTime="4.862076898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:43.744790215 +0000 UTC m=+499.406584573" watchObservedRunningTime="2026-07-10 01:08:43.746052656 +0000 UTC m=+499.407846988" Jul 10 01:08:43.762031 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.761973 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cd866567d-p5rjl" podStartSLOduration=2.612453896 podStartE2EDuration="8.761956918s" totalImagesPullingTime="6.149503022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:43.760344333 +0000 UTC m=+499.422138691" watchObservedRunningTime="2026-07-10 01:08:43.761956918 +0000 UTC m=+499.423751273" Jul 10 01:08:43.779687 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:43.779637 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" podStartSLOduration=1.7321135330000001 podStartE2EDuration="6.779618496s" totalImagesPullingTime="5.047504963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:43.77835096 +0000 UTC m=+499.440145317" watchObservedRunningTime="2026-07-10 01:08:43.779618496 +0000 UTC m=+499.441412851" Jul 10 01:08:44.714009 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:44.713982 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:44.969213 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:44.969128 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4"] Jul 10 01:08:44.972791 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:44.972764 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:44.983545 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:44.983518 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4"] Jul 10 01:08:45.005543 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005511 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005746 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8e60633-cdd4-449a-9d40-de71e4fd5702-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005746 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005746 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005665 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58kmn\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-kube-api-access-58kmn\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005746 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005746 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005713 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.005993 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.005770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106103 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106103 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106136 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106183 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8e60633-cdd4-449a-9d40-de71e4fd5702-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.106353 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58kmn\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-kube-api-access-58kmn\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.107011 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.106972 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.108950 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.108923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.109060 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.109023 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8e60633-cdd4-449a-9d40-de71e4fd5702-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.109156 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.109140 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.109249 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.109226 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.124002 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.123965 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.124137 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.124112 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58kmn\" (UniqueName: \"kubernetes.io/projected/e8e60633-cdd4-449a-9d40-de71e4fd5702-kube-api-access-58kmn\") pod \"istiod-openshift-gateway-94bb8fbfd-c88x4\" (UID: \"e8e60633-cdd4-449a-9d40-de71e4fd5702\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.282828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.282739 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.429810 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.429726 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4"] Jul 10 01:08:45.432182 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:45.432155 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e60633_cdd4_449a_9d40_de71e4fd5702.slice/crio-1a00cbcab55df27d96b59484df6ed53381e1fcdd2478b8fe308859f2fa8f8a7f WatchSource:0}: Error finding container 1a00cbcab55df27d96b59484df6ed53381e1fcdd2478b8fe308859f2fa8f8a7f: Status 404 returned error can't find the container with id 1a00cbcab55df27d96b59484df6ed53381e1fcdd2478b8fe308859f2fa8f8a7f Jul 10 01:08:45.435260 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.435228 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 01:08:45.435369 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.435288 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 01:08:45.719782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.719730 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" event={"ID":"e8e60633-cdd4-449a-9d40-de71e4fd5702","Type":"ContainerStarted","Data":"5946fcc23fb718afd729cd08b3f9ad030a6cce0c89724abec95317e1d1828c86"} Jul 10 01:08:45.720274 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.719796 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" event={"ID":"e8e60633-cdd4-449a-9d40-de71e4fd5702","Type":"ContainerStarted","Data":"1a00cbcab55df27d96b59484df6ed53381e1fcdd2478b8fe308859f2fa8f8a7f"} Jul 10 01:08:45.720274 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.719867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:45.743386 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:45.743329 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" podStartSLOduration=1.743307872 podStartE2EDuration="1.743307872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:45.741487487 +0000 UTC m=+501.403281841" watchObservedRunningTime="2026-07-10 01:08:45.743307872 +0000 UTC m=+501.405102228" Jul 10 01:08:46.727773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:46.727738 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-c88x4" Jul 10 01:08:46.787070 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:46.787030 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:46.787313 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:46.787262 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerName="discovery" containerID="cri-o://2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549" gracePeriod=30 Jul 10 01:08:47.035248 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.035219 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:47.123737 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123703 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123762 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123795 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123819 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123872 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-982qg\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123933 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.123968 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.123958 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig\") pod \"e9dac652-2c60-40c8-a115-36f9c378a36b\" (UID: \"e9dac652-2c60-40c8-a115-36f9c378a36b\") " Jul 10 01:08:47.124279 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.124249 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:08:47.126394 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126356 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts" (OuterVolumeSpecName: "cacerts") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:08:47.126394 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126379 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:08:47.126578 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126561 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs" (OuterVolumeSpecName: "local-certs") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:08:47.126677 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126649 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:08:47.126777 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126702 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:08:47.126869 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.126851 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg" (OuterVolumeSpecName: "kube-api-access-982qg") pod "e9dac652-2c60-40c8-a115-36f9c378a36b" (UID: "e9dac652-2c60-40c8-a115-36f9c378a36b"). InnerVolumeSpecName "kube-api-access-982qg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:08:47.224924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224869 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-ca-configmap\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.224924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224925 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9dac652-2c60-40c8-a115-36f9c378a36b-local-certs\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.225121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224939 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-token\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.225121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224948 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-cacerts\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.225121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224957 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-982qg\" (UniqueName: \"kubernetes.io/projected/e9dac652-2c60-40c8-a115-36f9c378a36b-kube-api-access-982qg\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.225121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224965 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-csr-dns-cert\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.225121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.224973 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9dac652-2c60-40c8-a115-36f9c378a36b-istio-kubeconfig\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:08:47.576143 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.576108 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r"] Jul 10 01:08:47.576465 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.576451 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerName="discovery" Jul 10 01:08:47.576511 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.576467 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerName="discovery" Jul 10 01:08:47.576545 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.576520 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerName="discovery" Jul 10 01:08:47.579162 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.579146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.583058 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.583028 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 01:08:47.583204 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.583145 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nfwgs\"" Jul 10 01:08:47.593925 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.593870 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r"] Jul 10 01:08:47.627988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.627950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.627988 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.627988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbphf\" (UniqueName: \"kubernetes.io/projected/c957deb5-7765-4339-860b-41f7b071d17c-kube-api-access-nbphf\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.729181 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.729149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.729181 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.729182 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbphf\" (UniqueName: \"kubernetes.io/projected/c957deb5-7765-4339-860b-41f7b071d17c-kube-api-access-nbphf\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.729703 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:08:47.729323 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 01:08:47.729703 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:08:47.729399 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert podName:c957deb5-7765-4339-860b-41f7b071d17c nodeName:}" failed. No retries permitted until 2026-07-10 01:08:48.229379143 +0000 UTC m=+503.891173476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert") pod "kserve-controller-manager-d6f4b65b8-dtb2r" (UID: "c957deb5-7765-4339-860b-41f7b071d17c") : secret "kserve-webhook-server-cert" not found Jul 10 01:08:47.731791 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.731768 2565 generic.go:356] "Generic (PLEG): container finished" podID="e9dac652-2c60-40c8-a115-36f9c378a36b" containerID="2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549" exitCode=0 Jul 10 01:08:47.731959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.731841 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" Jul 10 01:08:47.731959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.731834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" event={"ID":"e9dac652-2c60-40c8-a115-36f9c378a36b","Type":"ContainerDied","Data":"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549"} Jul 10 01:08:47.731959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.731947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4r48" event={"ID":"e9dac652-2c60-40c8-a115-36f9c378a36b","Type":"ContainerDied","Data":"f23df7e5a36708f05e13da013325b22ecff4a7c650f321066ea9a816b7c4133a"} Jul 10 01:08:47.732112 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.731975 2565 scope.go:117] "RemoveContainer" containerID="2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549" Jul 10 01:08:47.738982 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.738953 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbphf\" (UniqueName: \"kubernetes.io/projected/c957deb5-7765-4339-860b-41f7b071d17c-kube-api-access-nbphf\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:47.740726 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.740709 2565 scope.go:117] "RemoveContainer" containerID="2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549" Jul 10 01:08:47.741049 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:08:47.741028 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549\": container with ID starting with 2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549 not found: ID does not exist" containerID="2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549" Jul 10 01:08:47.741132 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.741060 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549"} err="failed to get container status \"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549\": rpc error: code = NotFound desc = could not find container \"2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549\": container with ID starting with 2cf3091517d55bd13733ffe624608959908f3f38c0957ce657ef300202668549 not found: ID does not exist" Jul 10 01:08:47.761906 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.761860 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:47.774690 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:47.774657 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4r48"] Jul 10 01:08:48.233137 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.233090 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:48.235714 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.235692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c957deb5-7765-4339-860b-41f7b071d17c-cert\") pod \"kserve-controller-manager-d6f4b65b8-dtb2r\" (UID: \"c957deb5-7765-4339-860b-41f7b071d17c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:48.489320 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.489238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:48.635927 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.635874 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r"] Jul 10 01:08:48.639135 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:08:48.639104 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc957deb5_7765_4339_860b_41f7b071d17c.slice/crio-e14f482ed0f9791087d912ba7d11ee825567cb40aae29fb7cf71a3459fb64086 WatchSource:0}: Error finding container e14f482ed0f9791087d912ba7d11ee825567cb40aae29fb7cf71a3459fb64086: Status 404 returned error can't find the container with id e14f482ed0f9791087d912ba7d11ee825567cb40aae29fb7cf71a3459fb64086 Jul 10 01:08:48.738550 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.738515 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" event={"ID":"c957deb5-7765-4339-860b-41f7b071d17c","Type":"ContainerStarted","Data":"e14f482ed0f9791087d912ba7d11ee825567cb40aae29fb7cf71a3459fb64086"} Jul 10 01:08:48.942623 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:48.942591 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9dac652-2c60-40c8-a115-36f9c378a36b" path="/var/lib/kubelet/pods/e9dac652-2c60-40c8-a115-36f9c378a36b/volumes" Jul 10 01:08:51.754604 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:51.754501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" event={"ID":"c957deb5-7765-4339-860b-41f7b071d17c","Type":"ContainerStarted","Data":"15630fda24801fde5e43da8b8f9ba08f5c3e12b75c9b6037444f0390675f6a8c"} Jul 10 01:08:51.754604 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:51.754570 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:08:51.775642 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:51.775589 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" podStartSLOduration=1.9185491510000001 podStartE2EDuration="4.775570619s" totalImagesPullingTime="2.857021468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:08:51.774279671 +0000 UTC m=+507.436074026" watchObservedRunningTime="2026-07-10 01:08:51.775570619 +0000 UTC m=+507.437364974" Jul 10 01:08:54.715358 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:54.715328 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hqzx2" Jul 10 01:08:54.717390 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:08:54.717369 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-b96gk" Jul 10 01:09:22.765735 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:22.765646 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dtb2r" Jul 10 01:09:37.116192 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.116160 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:37.121395 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.121367 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:37.124281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.124247 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nqmvn\"" Jul 10 01:09:37.125303 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.125280 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 01:09:37.125439 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.125279 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 01:09:37.130394 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.130364 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:37.137124 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.137097 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvv42\" (UniqueName: \"kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42\") pod \"kuadrant-operator-catalog-5rbfg\" (UID: \"a3eaac96-e3f0-440c-b39f-9cd4244c2274\") " pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:37.238006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.237971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvv42\" (UniqueName: \"kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42\") pod \"kuadrant-operator-catalog-5rbfg\" (UID: \"a3eaac96-e3f0-440c-b39f-9cd4244c2274\") " pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:37.248963 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.248931 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvv42\" (UniqueName: \"kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42\") pod \"kuadrant-operator-catalog-5rbfg\" (UID: \"a3eaac96-e3f0-440c-b39f-9cd4244c2274\") " pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:37.432942 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.432819 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:37.482236 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.482200 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:37.574464 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.574424 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:37.576727 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:37.576697 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3eaac96_e3f0_440c_b39f_9cd4244c2274.slice/crio-5619e90f78933988e93eb433039f201e4fcd0e00edda49ce87e1401aa87b2912 WatchSource:0}: Error finding container 5619e90f78933988e93eb433039f201e4fcd0e00edda49ce87e1401aa87b2912: Status 404 returned error can't find the container with id 5619e90f78933988e93eb433039f201e4fcd0e00edda49ce87e1401aa87b2912 Jul 10 01:09:37.692719 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.692631 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dszz9"] Jul 10 01:09:37.697019 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.696999 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:37.702971 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.702941 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dszz9"] Jul 10 01:09:37.743685 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.743633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z45tx\" (UniqueName: \"kubernetes.io/projected/267daf1f-9dd8-47d7-885e-a687f2ca1710-kube-api-access-z45tx\") pod \"kuadrant-operator-catalog-dszz9\" (UID: \"267daf1f-9dd8-47d7-885e-a687f2ca1710\") " pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:37.844767 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.844725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z45tx\" (UniqueName: \"kubernetes.io/projected/267daf1f-9dd8-47d7-885e-a687f2ca1710-kube-api-access-z45tx\") pod \"kuadrant-operator-catalog-dszz9\" (UID: \"267daf1f-9dd8-47d7-885e-a687f2ca1710\") " pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:37.852700 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.852664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z45tx\" (UniqueName: \"kubernetes.io/projected/267daf1f-9dd8-47d7-885e-a687f2ca1710-kube-api-access-z45tx\") pod \"kuadrant-operator-catalog-dszz9\" (UID: \"267daf1f-9dd8-47d7-885e-a687f2ca1710\") " pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:37.961320 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:37.961226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" event={"ID":"a3eaac96-e3f0-440c-b39f-9cd4244c2274","Type":"ContainerStarted","Data":"5619e90f78933988e93eb433039f201e4fcd0e00edda49ce87e1401aa87b2912"} Jul 10 01:09:38.008591 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:38.008559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:38.146275 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:38.146251 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dszz9"] Jul 10 01:09:38.148115 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:38.148088 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267daf1f_9dd8_47d7_885e_a687f2ca1710.slice/crio-48aeb4e34b9c5f205313acea2cf4f6e19370e73864c9dca2f62b515b8f7f8886 WatchSource:0}: Error finding container 48aeb4e34b9c5f205313acea2cf4f6e19370e73864c9dca2f62b515b8f7f8886: Status 404 returned error can't find the container with id 48aeb4e34b9c5f205313acea2cf4f6e19370e73864c9dca2f62b515b8f7f8886 Jul 10 01:09:38.968401 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:38.968361 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" event={"ID":"267daf1f-9dd8-47d7-885e-a687f2ca1710","Type":"ContainerStarted","Data":"48aeb4e34b9c5f205313acea2cf4f6e19370e73864c9dca2f62b515b8f7f8886"} Jul 10 01:09:39.975061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:39.975005 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" event={"ID":"267daf1f-9dd8-47d7-885e-a687f2ca1710","Type":"ContainerStarted","Data":"4c9cd97fe7c6dfd8cbee49c9a8c816be7fe9f7e04cd4643915c1487f82e18951"} Jul 10 01:09:39.976670 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:39.976640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" event={"ID":"a3eaac96-e3f0-440c-b39f-9cd4244c2274","Type":"ContainerStarted","Data":"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd"} Jul 10 01:09:39.976812 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:39.976745 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" podUID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" containerName="registry-server" containerID="cri-o://b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd" gracePeriod=2 Jul 10 01:09:39.990127 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:39.990069 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" podStartSLOduration=1.328749715 podStartE2EDuration="2.990054415s" totalImagesPullingTime="1.6613047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:09:39.98960496 +0000 UTC m=+555.651399327" watchObservedRunningTime="2026-07-10 01:09:39.990054415 +0000 UTC m=+555.651848749" Jul 10 01:09:40.005105 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.005048 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" podStartSLOduration=0.77996944 podStartE2EDuration="3.005029916s" totalImagesPullingTime="2.225060476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:09:40.002713434 +0000 UTC m=+555.664507791" watchObservedRunningTime="2026-07-10 01:09:40.005029916 +0000 UTC m=+555.666824271" Jul 10 01:09:40.221612 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.221583 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:40.266449 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.266407 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvv42\" (UniqueName: \"kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42\") pod \"a3eaac96-e3f0-440c-b39f-9cd4244c2274\" (UID: \"a3eaac96-e3f0-440c-b39f-9cd4244c2274\") " Jul 10 01:09:40.268757 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.268727 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42" (OuterVolumeSpecName: "kube-api-access-cvv42") pod "a3eaac96-e3f0-440c-b39f-9cd4244c2274" (UID: "a3eaac96-e3f0-440c-b39f-9cd4244c2274"). InnerVolumeSpecName "kube-api-access-cvv42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:09:40.367342 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.367254 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvv42\" (UniqueName: \"kubernetes.io/projected/a3eaac96-e3f0-440c-b39f-9cd4244c2274-kube-api-access-cvv42\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:40.982396 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.982361 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" containerID="b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd" exitCode=0 Jul 10 01:09:40.982801 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.982467 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" Jul 10 01:09:40.982801 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.982492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" event={"ID":"a3eaac96-e3f0-440c-b39f-9cd4244c2274","Type":"ContainerDied","Data":"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd"} Jul 10 01:09:40.982801 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.982541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rbfg" event={"ID":"a3eaac96-e3f0-440c-b39f-9cd4244c2274","Type":"ContainerDied","Data":"5619e90f78933988e93eb433039f201e4fcd0e00edda49ce87e1401aa87b2912"} Jul 10 01:09:40.982801 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.982562 2565 scope.go:117] "RemoveContainer" containerID="b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd" Jul 10 01:09:40.992126 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.992107 2565 scope.go:117] "RemoveContainer" containerID="b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd" Jul 10 01:09:40.992433 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:09:40.992411 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd\": container with ID starting with b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd not found: ID does not exist" containerID="b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd" Jul 10 01:09:40.992497 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.992446 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd"} err="failed to get container status \"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd\": rpc error: code = NotFound desc = could not find container \"b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd\": container with ID starting with b32241b477170f4592d8c8083faf357e9cfc03055fc6589fcd24290f98869bcd not found: ID does not exist" Jul 10 01:09:40.998982 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:40.998956 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:41.002463 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:41.002426 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rbfg"] Jul 10 01:09:42.942447 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:42.942416 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" path="/var/lib/kubelet/pods/a3eaac96-e3f0-440c-b39f-9cd4244c2274/volumes" Jul 10 01:09:48.009409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:48.009363 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:48.009842 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:48.009440 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:48.033087 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:48.033060 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:48.056042 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:48.056014 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dszz9" Jul 10 01:09:49.336277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.336243 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds"] Jul 10 01:09:49.336669 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.336546 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" containerName="registry-server" Jul 10 01:09:49.336669 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.336557 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" containerName="registry-server" Jul 10 01:09:49.336669 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.336633 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3eaac96-e3f0-440c-b39f-9cd4244c2274" containerName="registry-server" Jul 10 01:09:49.341782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.341748 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.345593 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.345571 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r8cm9\"" Jul 10 01:09:49.349134 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.349104 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds"] Jul 10 01:09:49.433299 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.433260 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.433473 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.433310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.433473 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.433339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfrj5\" (UniqueName: \"kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.534508 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.534472 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.534664 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.534516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.534664 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.534541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfrj5\" (UniqueName: \"kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.535006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.534984 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.535046 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.534982 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.543790 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.543762 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfrj5\" (UniqueName: \"kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.652863 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.652777 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:49.788041 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.787913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds"] Jul 10 01:09:49.790997 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:49.790965 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb772923b_4b26_43ca_9596_deb8631bdeae.slice/crio-d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168 WatchSource:0}: Error finding container d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168: Status 404 returned error can't find the container with id d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168 Jul 10 01:09:49.930372 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.930297 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c"] Jul 10 01:09:49.933952 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.933929 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:49.942409 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:49.942384 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c"] Jul 10 01:09:50.027878 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.027842 2565 generic.go:356] "Generic (PLEG): container finished" podID="b772923b-4b26-43ca-9596-deb8631bdeae" containerID="ff23b74bcaf576e0f3c67ac57a09c063c3ebf7c822d15bac8a31424074037c3f" exitCode=0 Jul 10 01:09:50.028057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.027941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" event={"ID":"b772923b-4b26-43ca-9596-deb8631bdeae","Type":"ContainerDied","Data":"ff23b74bcaf576e0f3c67ac57a09c063c3ebf7c822d15bac8a31424074037c3f"} Jul 10 01:09:50.028057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.027992 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" event={"ID":"b772923b-4b26-43ca-9596-deb8631bdeae","Type":"ContainerStarted","Data":"d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168"} Jul 10 01:09:50.038762 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.038734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nt6g\" (UniqueName: \"kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.038919 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.038774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.038919 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.038802 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.139959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.139917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nt6g\" (UniqueName: \"kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.139959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.139964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.140219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.139991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.140424 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.140401 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.140504 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.140413 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.148689 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.148661 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nt6g\" (UniqueName: \"kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.257343 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.257255 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:50.419318 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.419292 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c"] Jul 10 01:09:50.421680 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:50.421654 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb583c9_2562_4ab8_bd86_335e926855f5.slice/crio-4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7 WatchSource:0}: Error finding container 4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7: Status 404 returned error can't find the container with id 4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7 Jul 10 01:09:50.527165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.527135 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7"] Jul 10 01:09:50.530542 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.530523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.537501 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.537473 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7"] Jul 10 01:09:50.644352 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.644309 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sb96\" (UniqueName: \"kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.644523 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.644429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.644523 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.644507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.745340 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.745303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.745520 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.745355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.745520 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.745376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sb96\" (UniqueName: \"kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.745774 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.745751 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.745774 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.745767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.753836 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.753804 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sb96\" (UniqueName: \"kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:50.885114 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:50.885083 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:51.026077 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.026046 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7"] Jul 10 01:09:51.027853 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:51.027827 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47a2b76_4b9a_4f85_bf40_91142ea99bfa.slice/crio-e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49 WatchSource:0}: Error finding container e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49: Status 404 returned error can't find the container with id e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49 Jul 10 01:09:51.034946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.034862 2565 generic.go:356] "Generic (PLEG): container finished" podID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerID="64ff2386fde7dbf167bb0aea6f17b4cb1657f97714e3844398b3e98751479cec" exitCode=0 Jul 10 01:09:51.035057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.034945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" event={"ID":"bdb583c9-2562-4ab8-bd86-335e926855f5","Type":"ContainerDied","Data":"64ff2386fde7dbf167bb0aea6f17b4cb1657f97714e3844398b3e98751479cec"} Jul 10 01:09:51.035057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.034995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" event={"ID":"bdb583c9-2562-4ab8-bd86-335e926855f5","Type":"ContainerStarted","Data":"4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7"} Jul 10 01:09:51.036331 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.036304 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" event={"ID":"b47a2b76-4b9a-4f85-bf40-91142ea99bfa","Type":"ContainerStarted","Data":"e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49"} Jul 10 01:09:51.038322 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.038299 2565 generic.go:356] "Generic (PLEG): container finished" podID="b772923b-4b26-43ca-9596-deb8631bdeae" containerID="6732ea87865f2643806c6ee728ee0adb0e7144395744e0836cf19ce841be91aa" exitCode=0 Jul 10 01:09:51.038465 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.038342 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" event={"ID":"b772923b-4b26-43ca-9596-deb8631bdeae","Type":"ContainerDied","Data":"6732ea87865f2643806c6ee728ee0adb0e7144395744e0836cf19ce841be91aa"} Jul 10 01:09:51.130710 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.130683 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k"] Jul 10 01:09:51.134620 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.134593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.144874 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.144811 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k"] Jul 10 01:09:51.250784 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.250754 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.250964 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.250832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.250964 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.250868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn57k\" (UniqueName: \"kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.352282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.352247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.352438 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.352308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn57k\" (UniqueName: \"kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.352438 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.352354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.352752 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.352727 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.352784 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.352756 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.361006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.360976 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn57k\" (UniqueName: \"kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.532416 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.532377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:51.669514 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:51.669489 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k"] Jul 10 01:09:51.672171 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:09:51.672146 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5000a8a0_1351_4d2e_b4c1_510c56135580.slice/crio-cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd WatchSource:0}: Error finding container cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd: Status 404 returned error can't find the container with id cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd Jul 10 01:09:52.045548 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.045509 2565 generic.go:356] "Generic (PLEG): container finished" podID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerID="74aa604c9d6474d31f4768b43ee1a608b2b869b0888536a24a63e1975457f221" exitCode=0 Jul 10 01:09:52.045723 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.045597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" event={"ID":"bdb583c9-2562-4ab8-bd86-335e926855f5","Type":"ContainerDied","Data":"74aa604c9d6474d31f4768b43ee1a608b2b869b0888536a24a63e1975457f221"} Jul 10 01:09:52.047246 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.047193 2565 generic.go:356] "Generic (PLEG): container finished" podID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerID="b6dd4aed8b2aadb80e7ee601465fdd9533f9cf8980cbd54434b767f49a55233f" exitCode=0 Jul 10 01:09:52.047313 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.047277 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" event={"ID":"b47a2b76-4b9a-4f85-bf40-91142ea99bfa","Type":"ContainerDied","Data":"b6dd4aed8b2aadb80e7ee601465fdd9533f9cf8980cbd54434b767f49a55233f"} Jul 10 01:09:52.048868 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.048843 2565 generic.go:356] "Generic (PLEG): container finished" podID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerID="ef3b8a1df26f19e34e3eeb5a5c41e8e6fdfa24c53f3d80c6d47cc40a245deb03" exitCode=0 Jul 10 01:09:52.049006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.048945 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" event={"ID":"5000a8a0-1351-4d2e-b4c1-510c56135580","Type":"ContainerDied","Data":"ef3b8a1df26f19e34e3eeb5a5c41e8e6fdfa24c53f3d80c6d47cc40a245deb03"} Jul 10 01:09:52.049006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.048990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" event={"ID":"5000a8a0-1351-4d2e-b4c1-510c56135580","Type":"ContainerStarted","Data":"cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd"} Jul 10 01:09:52.051646 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.051545 2565 generic.go:356] "Generic (PLEG): container finished" podID="b772923b-4b26-43ca-9596-deb8631bdeae" containerID="ddea68ed8b7eec820cfed6ab4d515ec6d7c5a93b11efad758a4d3718cf8b7ab2" exitCode=0 Jul 10 01:09:52.051646 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:52.051605 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" event={"ID":"b772923b-4b26-43ca-9596-deb8631bdeae","Type":"ContainerDied","Data":"ddea68ed8b7eec820cfed6ab4d515ec6d7c5a93b11efad758a4d3718cf8b7ab2"} Jul 10 01:09:53.058359 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.058323 2565 generic.go:356] "Generic (PLEG): container finished" podID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerID="71891a5ba3ab56ac1690d1c5b668b1a75571092f3af7d234aa60916bb2f9c449" exitCode=0 Jul 10 01:09:53.058762 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.058411 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" event={"ID":"bdb583c9-2562-4ab8-bd86-335e926855f5","Type":"ContainerDied","Data":"71891a5ba3ab56ac1690d1c5b668b1a75571092f3af7d234aa60916bb2f9c449"} Jul 10 01:09:53.281158 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.281133 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:53.366825 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.366796 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle\") pod \"b772923b-4b26-43ca-9596-deb8631bdeae\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " Jul 10 01:09:53.367018 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.366879 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util\") pod \"b772923b-4b26-43ca-9596-deb8631bdeae\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " Jul 10 01:09:53.367018 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.366926 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfrj5\" (UniqueName: \"kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5\") pod \"b772923b-4b26-43ca-9596-deb8631bdeae\" (UID: \"b772923b-4b26-43ca-9596-deb8631bdeae\") " Jul 10 01:09:53.367491 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.367447 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle" (OuterVolumeSpecName: "bundle") pod "b772923b-4b26-43ca-9596-deb8631bdeae" (UID: "b772923b-4b26-43ca-9596-deb8631bdeae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:53.369069 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.369036 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5" (OuterVolumeSpecName: "kube-api-access-lfrj5") pod "b772923b-4b26-43ca-9596-deb8631bdeae" (UID: "b772923b-4b26-43ca-9596-deb8631bdeae"). InnerVolumeSpecName "kube-api-access-lfrj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:09:53.375293 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.375256 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util" (OuterVolumeSpecName: "util") pod "b772923b-4b26-43ca-9596-deb8631bdeae" (UID: "b772923b-4b26-43ca-9596-deb8631bdeae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:53.468026 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.467990 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfrj5\" (UniqueName: \"kubernetes.io/projected/b772923b-4b26-43ca-9596-deb8631bdeae-kube-api-access-lfrj5\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:53.468026 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.468020 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:53.468026 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:53.468029 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b772923b-4b26-43ca-9596-deb8631bdeae-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:54.067924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.067870 2565 generic.go:356] "Generic (PLEG): container finished" podID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerID="997819ea42da58239f64d28b2f55f79e5c71b677bff60cc4be6ed44c65a21e56" exitCode=0 Jul 10 01:09:54.068404 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.067987 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" event={"ID":"5000a8a0-1351-4d2e-b4c1-510c56135580","Type":"ContainerDied","Data":"997819ea42da58239f64d28b2f55f79e5c71b677bff60cc4be6ed44c65a21e56"} Jul 10 01:09:54.070053 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.070025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" event={"ID":"b772923b-4b26-43ca-9596-deb8631bdeae","Type":"ContainerDied","Data":"d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168"} Jul 10 01:09:54.070160 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.070051 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds" Jul 10 01:09:54.070160 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.070070 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8afc94c16b0273e1f0cae27629b34e054d71a6b1381be4a70b241bb11697168" Jul 10 01:09:54.071944 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.071918 2565 generic.go:356] "Generic (PLEG): container finished" podID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerID="71fe63fb011eb1b4ae2a39257b0219fa2ca2a6c07601416c8956c65df38b7bfb" exitCode=0 Jul 10 01:09:54.072038 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.071998 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" event={"ID":"b47a2b76-4b9a-4f85-bf40-91142ea99bfa","Type":"ContainerDied","Data":"71fe63fb011eb1b4ae2a39257b0219fa2ca2a6c07601416c8956c65df38b7bfb"} Jul 10 01:09:54.222720 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.222680 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:54.274848 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.274816 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util\") pod \"bdb583c9-2562-4ab8-bd86-335e926855f5\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " Jul 10 01:09:54.275057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.274867 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nt6g\" (UniqueName: \"kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g\") pod \"bdb583c9-2562-4ab8-bd86-335e926855f5\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " Jul 10 01:09:54.275057 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.274920 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle\") pod \"bdb583c9-2562-4ab8-bd86-335e926855f5\" (UID: \"bdb583c9-2562-4ab8-bd86-335e926855f5\") " Jul 10 01:09:54.275529 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.275505 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle" (OuterVolumeSpecName: "bundle") pod "bdb583c9-2562-4ab8-bd86-335e926855f5" (UID: "bdb583c9-2562-4ab8-bd86-335e926855f5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:54.277239 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.277212 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g" (OuterVolumeSpecName: "kube-api-access-5nt6g") pod "bdb583c9-2562-4ab8-bd86-335e926855f5" (UID: "bdb583c9-2562-4ab8-bd86-335e926855f5"). InnerVolumeSpecName "kube-api-access-5nt6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:09:54.280543 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.280518 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util" (OuterVolumeSpecName: "util") pod "bdb583c9-2562-4ab8-bd86-335e926855f5" (UID: "bdb583c9-2562-4ab8-bd86-335e926855f5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:54.375823 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.375788 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:54.375823 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.375822 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nt6g\" (UniqueName: \"kubernetes.io/projected/bdb583c9-2562-4ab8-bd86-335e926855f5-kube-api-access-5nt6g\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:54.376036 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:54.375831 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb583c9-2562-4ab8-bd86-335e926855f5-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:55.078236 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.078196 2565 generic.go:356] "Generic (PLEG): container finished" podID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerID="9dd470884b92b9c499fe27b55056a2ee272213b2d1899e69f5ceb69e272187ff" exitCode=0 Jul 10 01:09:55.078658 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.078289 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" event={"ID":"b47a2b76-4b9a-4f85-bf40-91142ea99bfa","Type":"ContainerDied","Data":"9dd470884b92b9c499fe27b55056a2ee272213b2d1899e69f5ceb69e272187ff"} Jul 10 01:09:55.080422 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.080386 2565 generic.go:356] "Generic (PLEG): container finished" podID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerID="cecb2f34f7b482fb3f5068d8ca4241563a22a97eaf1b9291e458353ce4ce40ee" exitCode=0 Jul 10 01:09:55.080551 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.080465 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" event={"ID":"5000a8a0-1351-4d2e-b4c1-510c56135580","Type":"ContainerDied","Data":"cecb2f34f7b482fb3f5068d8ca4241563a22a97eaf1b9291e458353ce4ce40ee"} Jul 10 01:09:55.082410 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.082388 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" event={"ID":"bdb583c9-2562-4ab8-bd86-335e926855f5","Type":"ContainerDied","Data":"4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7"} Jul 10 01:09:55.082524 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.082423 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ed6072c371d96deb766f55f590f43002fd6a3941ba8b33f8b7db4efa664ecf7" Jul 10 01:09:55.082524 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:55.082434 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c" Jul 10 01:09:56.224172 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.224145 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:56.253762 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.253731 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:56.290219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290181 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn57k\" (UniqueName: \"kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k\") pod \"5000a8a0-1351-4d2e-b4c1-510c56135580\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " Jul 10 01:09:56.290219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290218 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util\") pod \"5000a8a0-1351-4d2e-b4c1-510c56135580\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " Jul 10 01:09:56.290432 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290265 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util\") pod \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " Jul 10 01:09:56.290432 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290369 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sb96\" (UniqueName: \"kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96\") pod \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " Jul 10 01:09:56.290432 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290416 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle\") pod \"5000a8a0-1351-4d2e-b4c1-510c56135580\" (UID: \"5000a8a0-1351-4d2e-b4c1-510c56135580\") " Jul 10 01:09:56.290601 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290447 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle\") pod \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\" (UID: \"b47a2b76-4b9a-4f85-bf40-91142ea99bfa\") " Jul 10 01:09:56.290906 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.290863 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle" (OuterVolumeSpecName: "bundle") pod "5000a8a0-1351-4d2e-b4c1-510c56135580" (UID: "5000a8a0-1351-4d2e-b4c1-510c56135580"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:56.291193 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.291166 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle" (OuterVolumeSpecName: "bundle") pod "b47a2b76-4b9a-4f85-bf40-91142ea99bfa" (UID: "b47a2b76-4b9a-4f85-bf40-91142ea99bfa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:56.292628 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.292598 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96" (OuterVolumeSpecName: "kube-api-access-2sb96") pod "b47a2b76-4b9a-4f85-bf40-91142ea99bfa" (UID: "b47a2b76-4b9a-4f85-bf40-91142ea99bfa"). InnerVolumeSpecName "kube-api-access-2sb96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:09:56.292724 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.292622 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k" (OuterVolumeSpecName: "kube-api-access-zn57k") pod "5000a8a0-1351-4d2e-b4c1-510c56135580" (UID: "5000a8a0-1351-4d2e-b4c1-510c56135580"). InnerVolumeSpecName "kube-api-access-zn57k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:09:56.295883 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.295862 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util" (OuterVolumeSpecName: "util") pod "5000a8a0-1351-4d2e-b4c1-510c56135580" (UID: "5000a8a0-1351-4d2e-b4c1-510c56135580"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:56.360403 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.360296 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util" (OuterVolumeSpecName: "util") pod "b47a2b76-4b9a-4f85-bf40-91142ea99bfa" (UID: "b47a2b76-4b9a-4f85-bf40-91142ea99bfa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:09:56.391688 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391657 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sb96\" (UniqueName: \"kubernetes.io/projected/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-kube-api-access-2sb96\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:56.391688 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391688 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:56.391824 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391697 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-bundle\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:56.391824 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391706 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn57k\" (UniqueName: \"kubernetes.io/projected/5000a8a0-1351-4d2e-b4c1-510c56135580-kube-api-access-zn57k\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:56.391824 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391714 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5000a8a0-1351-4d2e-b4c1-510c56135580-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:56.391824 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:56.391722 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47a2b76-4b9a-4f85-bf40-91142ea99bfa-util\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:09:57.093501 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.093463 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" Jul 10 01:09:57.093501 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.093478 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7" event={"ID":"b47a2b76-4b9a-4f85-bf40-91142ea99bfa","Type":"ContainerDied","Data":"e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49"} Jul 10 01:09:57.093724 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.093530 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e53875ed1f532e2a9de10e9c37400507cee93d7e3c28fa5dc6d8ac581e5f7e49" Jul 10 01:09:57.095910 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.095482 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" event={"ID":"5000a8a0-1351-4d2e-b4c1-510c56135580","Type":"ContainerDied","Data":"cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd"} Jul 10 01:09:57.095910 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.095504 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k" Jul 10 01:09:57.095910 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:09:57.095528 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cabe37839d17e819cbe815d1a56918bf3c0bc01e1f6464facbeee9370bfb53fd" Jul 10 01:10:09.836950 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.836874 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9wncj"] Jul 10 01:10:09.837577 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837555 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="pull" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837581 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="pull" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837599 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="util" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837614 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="util" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837624 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="util" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837633 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="util" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837644 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="extract" Jul 10 01:10:09.837652 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837651 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837679 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="util" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837692 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="util" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837702 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837710 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837730 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837737 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837750 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837759 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837775 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837782 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837793 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="util" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837800 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="util" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837819 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837827 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837838 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837846 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="pull" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.837995 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b47a2b76-4b9a-4f85-bf40-91142ea99bfa" containerName="extract" Jul 10 01:10:09.838007 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.838012 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b772923b-4b26-43ca-9596-deb8631bdeae" containerName="extract" Jul 10 01:10:09.838743 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.838027 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5000a8a0-1351-4d2e-b4c1-510c56135580" containerName="extract" Jul 10 01:10:09.838743 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.838036 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb583c9-2562-4ab8-bd86-335e926855f5" containerName="extract" Jul 10 01:10:09.846577 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.846547 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9wncj"] Jul 10 01:10:09.846719 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.846642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:09.849630 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.849601 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hbgtj\"" Jul 10 01:10:09.908579 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:09.908542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjmmv\" (UniqueName: \"kubernetes.io/projected/1ca5ccdb-77bc-410a-93c8-513f2e133274-kube-api-access-mjmmv\") pod \"authorino-operator-75cc8d5cb7-9wncj\" (UID: \"1ca5ccdb-77bc-410a-93c8-513f2e133274\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:10.009720 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:10.009679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjmmv\" (UniqueName: \"kubernetes.io/projected/1ca5ccdb-77bc-410a-93c8-513f2e133274-kube-api-access-mjmmv\") pod \"authorino-operator-75cc8d5cb7-9wncj\" (UID: \"1ca5ccdb-77bc-410a-93c8-513f2e133274\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:10.017825 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:10.017791 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjmmv\" (UniqueName: \"kubernetes.io/projected/1ca5ccdb-77bc-410a-93c8-513f2e133274-kube-api-access-mjmmv\") pod \"authorino-operator-75cc8d5cb7-9wncj\" (UID: \"1ca5ccdb-77bc-410a-93c8-513f2e133274\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:10.158045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:10.157954 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:10.293761 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:10.293723 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9wncj"] Jul 10 01:10:10.296754 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:10:10.296719 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca5ccdb_77bc_410a_93c8_513f2e133274.slice/crio-868ae527b99594b0d81134c38ea320abbc81e058e1a05cd88cbb36e76bb65782 WatchSource:0}: Error finding container 868ae527b99594b0d81134c38ea320abbc81e058e1a05cd88cbb36e76bb65782: Status 404 returned error can't find the container with id 868ae527b99594b0d81134c38ea320abbc81e058e1a05cd88cbb36e76bb65782 Jul 10 01:10:11.161642 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:11.161598 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" event={"ID":"1ca5ccdb-77bc-410a-93c8-513f2e133274","Type":"ContainerStarted","Data":"868ae527b99594b0d81134c38ea320abbc81e058e1a05cd88cbb36e76bb65782"} Jul 10 01:10:13.173947 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:13.173910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" event={"ID":"1ca5ccdb-77bc-410a-93c8-513f2e133274","Type":"ContainerStarted","Data":"66f4797ef5c5532ce07dd8a1861aa49d304d55bcc881f206a46340ffed2e8e03"} Jul 10 01:10:13.174330 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:13.174030 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:13.193773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:13.193719 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" podStartSLOduration=1.88975495 podStartE2EDuration="4.193700267s" totalImagesPullingTime="2.303945317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:10:13.19186304 +0000 UTC m=+588.853657430" watchObservedRunningTime="2026-07-10 01:10:13.193700267 +0000 UTC m=+588.855494622" Jul 10 01:10:24.181129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:24.181096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9wncj" Jul 10 01:10:24.839205 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:24.839174 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:10:24.839453 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:24.839434 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:10:34.375908 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.375863 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj"] Jul 10 01:10:34.383371 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.383341 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.386166 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.386147 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8l2z5\"" Jul 10 01:10:34.393402 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.393372 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj"] Jul 10 01:10:34.522525 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.522480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxp4\" (UniqueName: \"kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.522525 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.522526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.623675 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.623630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.623854 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.623753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxp4\" (UniqueName: \"kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.624129 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.624104 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.634044 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.633993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxp4\" (UniqueName: \"kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.694717 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.694676 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:34.834921 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:34.834871 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj"] Jul 10 01:10:34.836960 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:10:34.836932 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfadddff7_17a1_416d_8493_39b85bc5b50c.slice/crio-b5f5a025771c8833b5890b0d07f72d1f411977911e6d7626bd8a1a8ae4956961 WatchSource:0}: Error finding container b5f5a025771c8833b5890b0d07f72d1f411977911e6d7626bd8a1a8ae4956961: Status 404 returned error can't find the container with id b5f5a025771c8833b5890b0d07f72d1f411977911e6d7626bd8a1a8ae4956961 Jul 10 01:10:35.234588 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.234551 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj"] Jul 10 01:10:35.241550 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.241521 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj"] Jul 10 01:10:35.261115 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.261083 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:10:35.265713 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.265689 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.279657 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.279572 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:10:35.430275 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.430230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zpbg\" (UniqueName: \"kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.430671 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.430315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.531251 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.531159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zpbg\" (UniqueName: \"kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.531251 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.531227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.531589 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.531570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.542539 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.542501 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zpbg\" (UniqueName: \"kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5sf6m\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.578379 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.578339 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:35.757658 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:35.757489 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:10:35.760219 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:10:35.760179 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89539cf3_ffa9_4378_84af_374baf83663c.slice/crio-e7627eb07a24737f50b62375f2c6d71fdf8bc01d6575ceab67e61e4c092a296e WatchSource:0}: Error finding container e7627eb07a24737f50b62375f2c6d71fdf8bc01d6575ceab67e61e4c092a296e: Status 404 returned error can't find the container with id e7627eb07a24737f50b62375f2c6d71fdf8bc01d6575ceab67e61e4c092a296e Jul 10 01:10:36.288134 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:36.288094 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" event={"ID":"89539cf3-ffa9-4378-84af-374baf83663c","Type":"ContainerStarted","Data":"e7627eb07a24737f50b62375f2c6d71fdf8bc01d6575ceab67e61e4c092a296e"} Jul 10 01:10:40.313310 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.313265 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" containerName="manager" containerID="cri-o://cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce" gracePeriod=2 Jul 10 01:10:40.315130 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.315100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" event={"ID":"89539cf3-ffa9-4378-84af-374baf83663c","Type":"ContainerStarted","Data":"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9"} Jul 10 01:10:40.315229 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.315215 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:10:40.334440 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.334394 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" podStartSLOduration=1.555736715 podStartE2EDuration="5.334375131s" totalImagesPullingTime="3.778638416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:10:40.332148869 +0000 UTC m=+615.993943228" watchObservedRunningTime="2026-07-10 01:10:40.334375131 +0000 UTC m=+615.996169486" Jul 10 01:10:40.552226 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.552193 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:40.554509 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:10:40.554470 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" is forbidden: User \"system:node:ip-10-0-128-136.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-136.ec2.internal' and this object" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:40.573861 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.573790 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvxp4\" (UniqueName: \"kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4\") pod \"fadddff7-17a1-416d-8493-39b85bc5b50c\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " Jul 10 01:10:40.573861 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.573826 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume\") pod \"fadddff7-17a1-416d-8493-39b85bc5b50c\" (UID: \"fadddff7-17a1-416d-8493-39b85bc5b50c\") " Jul 10 01:10:40.574154 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.574135 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fadddff7-17a1-416d-8493-39b85bc5b50c" (UID: "fadddff7-17a1-416d-8493-39b85bc5b50c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:10:40.576423 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.576396 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4" (OuterVolumeSpecName: "kube-api-access-wvxp4") pod "fadddff7-17a1-416d-8493-39b85bc5b50c" (UID: "fadddff7-17a1-416d-8493-39b85bc5b50c"). InnerVolumeSpecName "kube-api-access-wvxp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:10:40.674610 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.674574 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvxp4\" (UniqueName: \"kubernetes.io/projected/fadddff7-17a1-416d-8493-39b85bc5b50c-kube-api-access-wvxp4\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:10:40.674610 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.674605 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fadddff7-17a1-416d-8493-39b85bc5b50c-extensions-socket-volume\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:10:40.943320 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:40.943290 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" path="/var/lib/kubelet/pods/fadddff7-17a1-416d-8493-39b85bc5b50c/volumes" Jul 10 01:10:41.321101 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:41.321018 2565 generic.go:356] "Generic (PLEG): container finished" podID="fadddff7-17a1-416d-8493-39b85bc5b50c" containerID="cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce" exitCode=2 Jul 10 01:10:41.321101 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:41.321065 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:41.321571 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:41.321120 2565 scope.go:117] "RemoveContainer" containerID="cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce" Jul 10 01:10:41.325676 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:10:41.325640 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pzmsj\" is forbidden: User \"system:node:ip-10-0-128-136.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-136.ec2.internal' and this object" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pzmsj" Jul 10 01:10:41.330317 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:41.330298 2565 scope.go:117] "RemoveContainer" containerID="cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce" Jul 10 01:10:41.330621 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:10:41.330599 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce\": container with ID starting with cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce not found: ID does not exist" containerID="cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce" Jul 10 01:10:41.330686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:41.330636 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce"} err="failed to get container status \"cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce\": rpc error: code = NotFound desc = could not find container \"cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce\": container with ID starting with cd7949bf73f2e34f7eb105f04fb807773a67271c73e63175a752ef86dcc1c2ce not found: ID does not exist" Jul 10 01:10:51.323837 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:10:51.323757 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:11:03.616196 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.616162 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:11:03.616760 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.616452 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" podUID="89539cf3-ffa9-4378-84af-374baf83663c" containerName="manager" containerID="cri-o://01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9" gracePeriod=10 Jul 10 01:11:03.887955 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.887878 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:11:03.963175 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.963133 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zpbg\" (UniqueName: \"kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg\") pod \"89539cf3-ffa9-4378-84af-374baf83663c\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " Jul 10 01:11:03.963349 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.963187 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume\") pod \"89539cf3-ffa9-4378-84af-374baf83663c\" (UID: \"89539cf3-ffa9-4378-84af-374baf83663c\") " Jul 10 01:11:03.963686 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.963659 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "89539cf3-ffa9-4378-84af-374baf83663c" (UID: "89539cf3-ffa9-4378-84af-374baf83663c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:11:03.965540 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:03.965515 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg" (OuterVolumeSpecName: "kube-api-access-5zpbg") pod "89539cf3-ffa9-4378-84af-374baf83663c" (UID: "89539cf3-ffa9-4378-84af-374baf83663c"). InnerVolumeSpecName "kube-api-access-5zpbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:11:04.064428 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.064373 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zpbg\" (UniqueName: \"kubernetes.io/projected/89539cf3-ffa9-4378-84af-374baf83663c-kube-api-access-5zpbg\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:11:04.064428 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.064423 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89539cf3-ffa9-4378-84af-374baf83663c-extensions-socket-volume\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:11:04.432052 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.432015 2565 generic.go:356] "Generic (PLEG): container finished" podID="89539cf3-ffa9-4378-84af-374baf83663c" containerID="01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9" exitCode=0 Jul 10 01:11:04.432254 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.432066 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" Jul 10 01:11:04.432254 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.432107 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" event={"ID":"89539cf3-ffa9-4378-84af-374baf83663c","Type":"ContainerDied","Data":"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9"} Jul 10 01:11:04.432254 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.432177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m" event={"ID":"89539cf3-ffa9-4378-84af-374baf83663c","Type":"ContainerDied","Data":"e7627eb07a24737f50b62375f2c6d71fdf8bc01d6575ceab67e61e4c092a296e"} Jul 10 01:11:04.432254 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.432209 2565 scope.go:117] "RemoveContainer" containerID="01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9" Jul 10 01:11:04.441605 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.441580 2565 scope.go:117] "RemoveContainer" containerID="01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9" Jul 10 01:11:04.441915 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:11:04.441876 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9\": container with ID starting with 01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9 not found: ID does not exist" containerID="01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9" Jul 10 01:11:04.442036 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.441923 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9"} err="failed to get container status \"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9\": rpc error: code = NotFound desc = could not find container \"01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9\": container with ID starting with 01219ae15678ff543e1ee2c110bbc2901bf0948f85f658b827f5fedaae53c4c9 not found: ID does not exist" Jul 10 01:11:04.453731 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.453700 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:11:04.457393 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.457365 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5sf6m"] Jul 10 01:11:04.943700 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:04.943668 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89539cf3-ffa9-4378-84af-374baf83663c" path="/var/lib/kubelet/pods/89539cf3-ffa9-4378-84af-374baf83663c/volumes" Jul 10 01:11:23.546456 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546416 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546725 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89539cf3-ffa9-4378-84af-374baf83663c" containerName="manager" Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546737 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="89539cf3-ffa9-4378-84af-374baf83663c" containerName="manager" Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546749 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" containerName="manager" Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546755 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" containerName="manager" Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546803 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fadddff7-17a1-416d-8493-39b85bc5b50c" containerName="manager" Jul 10 01:11:23.546969 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.546810 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="89539cf3-ffa9-4378-84af-374baf83663c" containerName="manager" Jul 10 01:11:23.549743 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.549722 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.552213 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.552185 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 01:11:23.552344 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.552326 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r8cm9\"" Jul 10 01:11:23.559308 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.559278 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:23.627434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.627378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfw9v\" (UniqueName: \"kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.627621 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.627545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.643811 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.643777 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:23.729231 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.729174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.729477 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.729280 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfw9v\" (UniqueName: \"kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.730102 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.730072 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.738052 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.738023 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfw9v\" (UniqueName: \"kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v\") pod \"limitador-limitador-64c94fd64f-mtg4f\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:23.861504 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:23.861457 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:24.017780 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.017753 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:24.019938 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:11:24.019909 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda066816_0ee5_4983_8180_ea19f08dd21a.slice/crio-c859fac030b5638e1d7cad19078c3aab9d012a5f4d1dc0c50d439007a9aef70f WatchSource:0}: Error finding container c859fac030b5638e1d7cad19078c3aab9d012a5f4d1dc0c50d439007a9aef70f: Status 404 returned error can't find the container with id c859fac030b5638e1d7cad19078c3aab9d012a5f4d1dc0c50d439007a9aef70f Jul 10 01:11:24.532695 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.532658 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" event={"ID":"da066816-0ee5-4983-8180-ea19f08dd21a","Type":"ContainerStarted","Data":"c859fac030b5638e1d7cad19078c3aab9d012a5f4d1dc0c50d439007a9aef70f"} Jul 10 01:11:24.696021 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.695986 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:11:24.700741 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.700718 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:11:24.703257 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.703233 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rqk9\"" Jul 10 01:11:24.709159 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.709131 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:11:24.839259 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.839221 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v46pz\" (UniqueName: \"kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz\") pod \"authorino-75d556bd77-8r68b\" (UID: \"e457d3c2-de3f-4bd8-a329-2f520b43a9f2\") " pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:11:24.940481 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.940448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v46pz\" (UniqueName: \"kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz\") pod \"authorino-75d556bd77-8r68b\" (UID: \"e457d3c2-de3f-4bd8-a329-2f520b43a9f2\") " pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:11:24.949103 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:24.949060 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v46pz\" (UniqueName: \"kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz\") pod \"authorino-75d556bd77-8r68b\" (UID: \"e457d3c2-de3f-4bd8-a329-2f520b43a9f2\") " pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:11:25.013401 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:25.013286 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rqk9\"" Jul 10 01:11:25.021187 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:25.021058 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:11:25.202118 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:25.202019 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:11:25.205281 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:11:25.205231 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode457d3c2_de3f_4bd8_a329_2f520b43a9f2.slice/crio-fad74b34e2a4d77612358f049da868749e17d6a5da2cb3266a2fc4e5368bfdb4 WatchSource:0}: Error finding container fad74b34e2a4d77612358f049da868749e17d6a5da2cb3266a2fc4e5368bfdb4: Status 404 returned error can't find the container with id fad74b34e2a4d77612358f049da868749e17d6a5da2cb3266a2fc4e5368bfdb4 Jul 10 01:11:25.540712 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:25.540619 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r68b" event={"ID":"e457d3c2-de3f-4bd8-a329-2f520b43a9f2","Type":"ContainerStarted","Data":"fad74b34e2a4d77612358f049da868749e17d6a5da2cb3266a2fc4e5368bfdb4"} Jul 10 01:11:27.561109 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:27.560936 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" event={"ID":"da066816-0ee5-4983-8180-ea19f08dd21a","Type":"ContainerStarted","Data":"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023"} Jul 10 01:11:27.561109 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:27.561084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:27.579757 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:27.579694 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" podStartSLOduration=1.552096055 podStartE2EDuration="4.579675203s" totalImagesPullingTime="3.027579148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:11:27.578748218 +0000 UTC m=+663.240542586" watchObservedRunningTime="2026-07-10 01:11:27.579675203 +0000 UTC m=+663.241469559" Jul 10 01:11:29.574766 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:29.574679 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r68b" event={"ID":"e457d3c2-de3f-4bd8-a329-2f520b43a9f2","Type":"ContainerStarted","Data":"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67"} Jul 10 01:11:29.590882 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:29.590824 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-8r68b" podStartSLOduration=1.53669718 podStartE2EDuration="5.590804553s" totalImagesPullingTime="4.054107373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:11:29.58891599 +0000 UTC m=+665.250710335" watchObservedRunningTime="2026-07-10 01:11:29.590804553 +0000 UTC m=+665.252598909" Jul 10 01:11:38.570183 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:38.570143 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:40.722568 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:40.722528 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:40.723077 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:40.722726 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" podUID="da066816-0ee5-4983-8180-ea19f08dd21a" containerName="limitador" containerID="cri-o://e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023" gracePeriod=30 Jul 10 01:11:41.281372 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.281345 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:41.372229 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.372194 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file\") pod \"da066816-0ee5-4983-8180-ea19f08dd21a\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " Jul 10 01:11:41.372229 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.372232 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfw9v\" (UniqueName: \"kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v\") pod \"da066816-0ee5-4983-8180-ea19f08dd21a\" (UID: \"da066816-0ee5-4983-8180-ea19f08dd21a\") " Jul 10 01:11:41.372570 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.372546 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file" (OuterVolumeSpecName: "config-file") pod "da066816-0ee5-4983-8180-ea19f08dd21a" (UID: "da066816-0ee5-4983-8180-ea19f08dd21a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:11:41.374477 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.374453 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v" (OuterVolumeSpecName: "kube-api-access-sfw9v") pod "da066816-0ee5-4983-8180-ea19f08dd21a" (UID: "da066816-0ee5-4983-8180-ea19f08dd21a"). InnerVolumeSpecName "kube-api-access-sfw9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:11:41.473726 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.473690 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da066816-0ee5-4983-8180-ea19f08dd21a-config-file\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:11:41.473726 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.473720 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfw9v\" (UniqueName: \"kubernetes.io/projected/da066816-0ee5-4983-8180-ea19f08dd21a-kube-api-access-sfw9v\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:11:41.633794 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.633705 2565 generic.go:356] "Generic (PLEG): container finished" podID="da066816-0ee5-4983-8180-ea19f08dd21a" containerID="e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023" exitCode=0 Jul 10 01:11:41.633794 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.633750 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" Jul 10 01:11:41.633794 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.633784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" event={"ID":"da066816-0ee5-4983-8180-ea19f08dd21a","Type":"ContainerDied","Data":"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023"} Jul 10 01:11:41.634058 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.633829 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mtg4f" event={"ID":"da066816-0ee5-4983-8180-ea19f08dd21a","Type":"ContainerDied","Data":"c859fac030b5638e1d7cad19078c3aab9d012a5f4d1dc0c50d439007a9aef70f"} Jul 10 01:11:41.634058 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.633852 2565 scope.go:117] "RemoveContainer" containerID="e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023" Jul 10 01:11:41.643108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.643083 2565 scope.go:117] "RemoveContainer" containerID="e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023" Jul 10 01:11:41.643402 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:11:41.643382 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023\": container with ID starting with e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023 not found: ID does not exist" containerID="e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023" Jul 10 01:11:41.643455 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.643412 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023"} err="failed to get container status \"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023\": rpc error: code = NotFound desc = could not find container \"e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023\": container with ID starting with e387b23f4e85b9e0a79cee4ea5258698c9dea30c91782566b025bd8bfd5f7023 not found: ID does not exist" Jul 10 01:11:41.655682 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.655648 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:41.657511 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:41.657490 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mtg4f"] Jul 10 01:11:42.943322 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:42.943286 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da066816-0ee5-4983-8180-ea19f08dd21a" path="/var/lib/kubelet/pods/da066816-0ee5-4983-8180-ea19f08dd21a/volumes" Jul 10 01:11:46.889075 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.889041 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj"] Jul 10 01:11:46.889520 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.889470 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da066816-0ee5-4983-8180-ea19f08dd21a" containerName="limitador" Jul 10 01:11:46.889520 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.889489 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="da066816-0ee5-4983-8180-ea19f08dd21a" containerName="limitador" Jul 10 01:11:46.889646 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.889585 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="da066816-0ee5-4983-8180-ea19f08dd21a" containerName="limitador" Jul 10 01:11:46.892413 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.892391 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:46.895518 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.895488 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-d4jvw\"" Jul 10 01:11:46.895518 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.895508 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 01:11:46.895702 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.895578 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 01:11:46.896482 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.896461 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 01:11:46.903222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.903202 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj"] Jul 10 01:11:46.917882 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.917851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f510aa4a-df1f-42b2-b75e-36b25cdd538f-data\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:46.917882 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:46.917884 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jcj\" (UniqueName: \"kubernetes.io/projected/f510aa4a-df1f-42b2-b75e-36b25cdd538f-kube-api-access-m8jcj\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.018959 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.018920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f510aa4a-df1f-42b2-b75e-36b25cdd538f-data\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.019125 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.018966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jcj\" (UniqueName: \"kubernetes.io/projected/f510aa4a-df1f-42b2-b75e-36b25cdd538f-kube-api-access-m8jcj\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.019245 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.019223 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f510aa4a-df1f-42b2-b75e-36b25cdd538f-data\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.028935 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.028912 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jcj\" (UniqueName: \"kubernetes.io/projected/f510aa4a-df1f-42b2-b75e-36b25cdd538f-kube-api-access-m8jcj\") pod \"postgres-69fb4594cf-j6tbj\" (UID: \"f510aa4a-df1f-42b2-b75e-36b25cdd538f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.203171 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.203090 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:47.346543 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.346519 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj"] Jul 10 01:11:47.349139 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:11:47.349109 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf510aa4a_df1f_42b2_b75e_36b25cdd538f.slice/crio-deb02a86983985d9eaf6ad5637ebc2a39d9c507bcdd48d8d1ec3780c2731e355 WatchSource:0}: Error finding container deb02a86983985d9eaf6ad5637ebc2a39d9c507bcdd48d8d1ec3780c2731e355: Status 404 returned error can't find the container with id deb02a86983985d9eaf6ad5637ebc2a39d9c507bcdd48d8d1ec3780c2731e355 Jul 10 01:11:47.350651 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.350627 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 01:11:47.664029 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:47.663995 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" event={"ID":"f510aa4a-df1f-42b2-b75e-36b25cdd538f","Type":"ContainerStarted","Data":"deb02a86983985d9eaf6ad5637ebc2a39d9c507bcdd48d8d1ec3780c2731e355"} Jul 10 01:11:52.692777 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:52.692682 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" event={"ID":"f510aa4a-df1f-42b2-b75e-36b25cdd538f","Type":"ContainerStarted","Data":"92be68e91eb9a3d137314ebb427e819090e6cfd443dc9da710d1bb8439209c9c"} Jul 10 01:11:52.692777 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:52.692760 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:11:52.707176 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:52.707116 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" podStartSLOduration=1.651638777 podStartE2EDuration="6.707093862s" totalImagesPullingTime="5.055455085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:11:52.706949649 +0000 UTC m=+688.368744020" watchObservedRunningTime="2026-07-10 01:11:52.707093862 +0000 UTC m=+688.368888220" Jul 10 01:11:58.728670 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:11:58.728637 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j6tbj" Jul 10 01:12:00.870049 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:00.870009 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:12:00.870542 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:00.870267 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-8r68b" podUID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" containerName="authorino" containerID="cri-o://354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67" gracePeriod=30 Jul 10 01:12:01.152723 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.152683 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:12:01.252617 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.252571 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v46pz\" (UniqueName: \"kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz\") pod \"e457d3c2-de3f-4bd8-a329-2f520b43a9f2\" (UID: \"e457d3c2-de3f-4bd8-a329-2f520b43a9f2\") " Jul 10 01:12:01.254799 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.254761 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz" (OuterVolumeSpecName: "kube-api-access-v46pz") pod "e457d3c2-de3f-4bd8-a329-2f520b43a9f2" (UID: "e457d3c2-de3f-4bd8-a329-2f520b43a9f2"). InnerVolumeSpecName "kube-api-access-v46pz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:12:01.353782 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.353743 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v46pz\" (UniqueName: \"kubernetes.io/projected/e457d3c2-de3f-4bd8-a329-2f520b43a9f2-kube-api-access-v46pz\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:12:01.738128 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.738096 2565 generic.go:356] "Generic (PLEG): container finished" podID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" containerID="354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67" exitCode=0 Jul 10 01:12:01.738297 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.738147 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r68b" Jul 10 01:12:01.738297 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.738187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r68b" event={"ID":"e457d3c2-de3f-4bd8-a329-2f520b43a9f2","Type":"ContainerDied","Data":"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67"} Jul 10 01:12:01.738297 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.738238 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r68b" event={"ID":"e457d3c2-de3f-4bd8-a329-2f520b43a9f2","Type":"ContainerDied","Data":"fad74b34e2a4d77612358f049da868749e17d6a5da2cb3266a2fc4e5368bfdb4"} Jul 10 01:12:01.738297 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.738259 2565 scope.go:117] "RemoveContainer" containerID="354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67" Jul 10 01:12:01.747851 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.747829 2565 scope.go:117] "RemoveContainer" containerID="354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67" Jul 10 01:12:01.748144 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:12:01.748127 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67\": container with ID starting with 354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67 not found: ID does not exist" containerID="354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67" Jul 10 01:12:01.748188 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.748155 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67"} err="failed to get container status \"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67\": rpc error: code = NotFound desc = could not find container \"354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67\": container with ID starting with 354fbea8063291d0f1ade466719b405a17caa43e2b2b61434203a3088ce26e67 not found: ID does not exist" Jul 10 01:12:01.759625 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.759587 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:12:01.764207 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:01.764180 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r68b"] Jul 10 01:12:02.943533 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:02.943499 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" path="/var/lib/kubelet/pods/e457d3c2-de3f-4bd8-a329-2f520b43a9f2/volumes" Jul 10 01:12:05.327576 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.327545 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:05.327961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.327850 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" containerName="authorino" Jul 10 01:12:05.327961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.327863 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" containerName="authorino" Jul 10 01:12:05.327961 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.327935 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e457d3c2-de3f-4bd8-a329-2f520b43a9f2" containerName="authorino" Jul 10 01:12:05.329781 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.329763 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.336113 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.336086 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 01:12:05.336283 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.336116 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 01:12:05.336676 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.336496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q7hpk\"" Jul 10 01:12:05.345434 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.345403 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:05.493940 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.493864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.494123 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.493953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vnww\" (UniqueName: \"kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.494123 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.494018 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.510781 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.510744 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:12:05.513186 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.513151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.525055 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.525028 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:12:05.595107 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.595265 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595130 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.595265 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.595265 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vnww\" (UniqueName: \"kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.595265 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595246 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rkns\" (UniqueName: \"kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.595410 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595271 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.595865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.595839 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.597679 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.597654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.602924 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.602876 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vnww\" (UniqueName: \"kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww\") pod \"maas-controller-76d66fc464-hk7dt\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.640973 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.640932 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:05.696427 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.696333 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.696585 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.696429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rkns\" (UniqueName: \"kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.696585 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.696480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.697008 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.696987 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.699144 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.699114 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.709132 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.708989 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rkns\" (UniqueName: \"kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns\") pod \"maas-controller-7867f4b576-w8mhc\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.779526 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.779498 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:05.781519 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:12:05.781488 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bff2824_f2b0_4ba7_b6f2_d5d1fe10f447.slice/crio-c438d8255e04948f04dbe543a781a98c71393183a2337821becf9896ea8e4446 WatchSource:0}: Error finding container c438d8255e04948f04dbe543a781a98c71393183a2337821becf9896ea8e4446: Status 404 returned error can't find the container with id c438d8255e04948f04dbe543a781a98c71393183a2337821becf9896ea8e4446 Jul 10 01:12:05.822443 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.822403 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:05.956172 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:05.956139 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:12:05.957944 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:12:05.957919 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a3c9f7_33ba_4d9f_95ff_0797e719adf0.slice/crio-c447271d024892a85dde279359e1df08d28ef01070bb6af4798e24bed32f6439 WatchSource:0}: Error finding container c447271d024892a85dde279359e1df08d28ef01070bb6af4798e24bed32f6439: Status 404 returned error can't find the container with id c447271d024892a85dde279359e1df08d28ef01070bb6af4798e24bed32f6439 Jul 10 01:12:06.774073 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:06.774026 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-w8mhc" event={"ID":"18a3c9f7-33ba-4d9f-95ff-0797e719adf0","Type":"ContainerStarted","Data":"c447271d024892a85dde279359e1df08d28ef01070bb6af4798e24bed32f6439"} Jul 10 01:12:06.776211 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:06.776164 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d66fc464-hk7dt" event={"ID":"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447","Type":"ContainerStarted","Data":"c438d8255e04948f04dbe543a781a98c71393183a2337821becf9896ea8e4446"} Jul 10 01:12:08.788130 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.788084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-w8mhc" event={"ID":"18a3c9f7-33ba-4d9f-95ff-0797e719adf0","Type":"ContainerStarted","Data":"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50"} Jul 10 01:12:08.788666 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.788205 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:08.789778 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.789749 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d66fc464-hk7dt" event={"ID":"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447","Type":"ContainerStarted","Data":"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0"} Jul 10 01:12:08.789984 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.789916 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:08.809047 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.808987 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7867f4b576-w8mhc" podStartSLOduration=1.8680985300000001 podStartE2EDuration="3.808961389s" totalImagesPullingTime="1.940862859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:12:08.806942073 +0000 UTC m=+704.468736430" watchObservedRunningTime="2026-07-10 01:12:08.808961389 +0000 UTC m=+704.470755746" Jul 10 01:12:08.825281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:08.825227 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d66fc464-hk7dt" podStartSLOduration=1.718727312 podStartE2EDuration="3.825209901s" totalImagesPullingTime="2.106482589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:12:08.822853852 +0000 UTC m=+704.484648209" watchObservedRunningTime="2026-07-10 01:12:08.825209901 +0000 UTC m=+704.487004255" Jul 10 01:12:19.799747 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:19.799657 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:12:19.799747 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:19.799729 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:19.847885 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:19.847849 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:19.848230 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:19.848168 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76d66fc464-hk7dt" podUID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" containerName="manager" containerID="cri-o://d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0" gracePeriod=30 Jul 10 01:12:20.091037 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.091006 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:20.128626 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.128585 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca\") pod \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " Jul 10 01:12:20.128810 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.128644 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert\") pod \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " Jul 10 01:12:20.128810 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.128693 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vnww\" (UniqueName: \"kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww\") pod \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\" (UID: \"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447\") " Jul 10 01:12:20.129006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.128981 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca" (OuterVolumeSpecName: "service-ca") pod "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" (UID: "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:12:20.131090 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.131063 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" (UID: "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:12:20.131167 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.131070 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww" (OuterVolumeSpecName: "kube-api-access-5vnww") pod "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" (UID: "7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447"). InnerVolumeSpecName "kube-api-access-5vnww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:12:20.229911 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.229863 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-service-ca\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:12:20.229911 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.229922 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-webhook-cert\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:12:20.230126 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.229932 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vnww\" (UniqueName: \"kubernetes.io/projected/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447-kube-api-access-5vnww\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:12:20.848759 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.848724 2565 generic.go:356] "Generic (PLEG): container finished" podID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" containerID="d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0" exitCode=0 Jul 10 01:12:20.849244 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.848765 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d66fc464-hk7dt" Jul 10 01:12:20.849244 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.848811 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d66fc464-hk7dt" event={"ID":"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447","Type":"ContainerDied","Data":"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0"} Jul 10 01:12:20.849244 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.848873 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d66fc464-hk7dt" event={"ID":"7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447","Type":"ContainerDied","Data":"c438d8255e04948f04dbe543a781a98c71393183a2337821becf9896ea8e4446"} Jul 10 01:12:20.849244 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.848928 2565 scope.go:117] "RemoveContainer" containerID="d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0" Jul 10 01:12:20.858255 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.858234 2565 scope.go:117] "RemoveContainer" containerID="d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0" Jul 10 01:12:20.858551 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:12:20.858533 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0\": container with ID starting with d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0 not found: ID does not exist" containerID="d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0" Jul 10 01:12:20.858611 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.858562 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0"} err="failed to get container status \"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0\": rpc error: code = NotFound desc = could not find container \"d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0\": container with ID starting with d110ce8d8fdc9a72e6befaf23367d0c16dbbbf3cc790e810f4d06e4a563e7ec0 not found: ID does not exist" Jul 10 01:12:20.870121 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.870096 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:20.873611 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.873588 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76d66fc464-hk7dt"] Jul 10 01:12:20.943208 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:20.943168 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" path="/var/lib/kubelet/pods/7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447/volumes" Jul 10 01:12:40.104133 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.104096 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-db47c776-n5c2h"] Jul 10 01:12:40.104603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.104413 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" containerName="manager" Jul 10 01:12:40.104603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.104423 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" containerName="manager" Jul 10 01:12:40.104603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.104485 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bff2824-f2b0-4ba7-b6f2-d5d1fe10f447" containerName="manager" Jul 10 01:12:40.130155 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.130118 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-db47c776-n5c2h"] Jul 10 01:12:40.130323 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.130249 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.135265 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.135243 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 01:12:40.136452 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.136415 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kcl6g\"" Jul 10 01:12:40.194828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.194777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1689f147-8298-4897-80a7-ccf30483d6b0-openshift-service-ca\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.194828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.194825 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1689f147-8298-4897-80a7-ccf30483d6b0-maas-api-tls\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.195102 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.194842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc6jh\" (UniqueName: \"kubernetes.io/projected/1689f147-8298-4897-80a7-ccf30483d6b0-kube-api-access-sc6jh\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.295498 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.295464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1689f147-8298-4897-80a7-ccf30483d6b0-openshift-service-ca\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.295694 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.295513 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1689f147-8298-4897-80a7-ccf30483d6b0-maas-api-tls\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.295694 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.295540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc6jh\" (UniqueName: \"kubernetes.io/projected/1689f147-8298-4897-80a7-ccf30483d6b0-kube-api-access-sc6jh\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.296165 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.296138 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1689f147-8298-4897-80a7-ccf30483d6b0-openshift-service-ca\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.298488 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.298462 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1689f147-8298-4897-80a7-ccf30483d6b0-maas-api-tls\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.304701 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.304669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc6jh\" (UniqueName: \"kubernetes.io/projected/1689f147-8298-4897-80a7-ccf30483d6b0-kube-api-access-sc6jh\") pod \"maas-api-db47c776-n5c2h\" (UID: \"1689f147-8298-4897-80a7-ccf30483d6b0\") " pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.440950 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.440829 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:40.589433 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.589402 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-db47c776-n5c2h"] Jul 10 01:12:40.592114 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:12:40.592083 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1689f147_8298_4897_80a7_ccf30483d6b0.slice/crio-171fa5816201748a5fdd69faa8fe6718668b13b93bdf337326a575ace87261de WatchSource:0}: Error finding container 171fa5816201748a5fdd69faa8fe6718668b13b93bdf337326a575ace87261de: Status 404 returned error can't find the container with id 171fa5816201748a5fdd69faa8fe6718668b13b93bdf337326a575ace87261de Jul 10 01:12:40.632497 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.632463 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cs9gs"] Jul 10 01:12:40.638645 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.638620 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.640987 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.640960 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 01:12:40.641133 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.641006 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n2l6s\"" Jul 10 01:12:40.645930 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.645877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cs9gs"] Jul 10 01:12:40.699162 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.699061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5t9w\" (UniqueName: \"kubernetes.io/projected/83c77c9d-64a8-4008-aa90-49f7a2f99a79-kube-api-access-h5t9w\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.699321 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.699188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c77c9d-64a8-4008-aa90-49f7a2f99a79-plugins-config-volume\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.703086 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.703057 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4cd8p"] Jul 10 01:12:40.707354 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.707333 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.716196 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.716166 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4cd8p"] Jul 10 01:12:40.800564 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.800529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44cda98b-bb58-4d4b-a044-1b5501d1927b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.800766 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.800577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5t9w\" (UniqueName: \"kubernetes.io/projected/83c77c9d-64a8-4008-aa90-49f7a2f99a79-kube-api-access-h5t9w\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.800766 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.800618 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqt9q\" (UniqueName: \"kubernetes.io/projected/44cda98b-bb58-4d4b-a044-1b5501d1927b-kube-api-access-vqt9q\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.800766 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.800692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c77c9d-64a8-4008-aa90-49f7a2f99a79-plugins-config-volume\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.801440 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.801413 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83c77c9d-64a8-4008-aa90-49f7a2f99a79-plugins-config-volume\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.812315 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.812275 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5t9w\" (UniqueName: \"kubernetes.io/projected/83c77c9d-64a8-4008-aa90-49f7a2f99a79-kube-api-access-h5t9w\") pod \"payload-processing-58f86c7798-cs9gs\" (UID: \"83c77c9d-64a8-4008-aa90-49f7a2f99a79\") " pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.901778 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.901734 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44cda98b-bb58-4d4b-a044-1b5501d1927b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.902006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.901791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqt9q\" (UniqueName: \"kubernetes.io/projected/44cda98b-bb58-4d4b-a044-1b5501d1927b-kube-api-access-vqt9q\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.902473 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.902452 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44cda98b-bb58-4d4b-a044-1b5501d1927b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.909632 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.909595 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqt9q\" (UniqueName: \"kubernetes.io/projected/44cda98b-bb58-4d4b-a044-1b5501d1927b-kube-api-access-vqt9q\") pod \"payload-pre-processing-7964457696-4cd8p\" (UID: \"44cda98b-bb58-4d4b-a044-1b5501d1927b\") " pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:40.951651 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.951567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:40.954092 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:40.954061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" event={"ID":"1689f147-8298-4897-80a7-ccf30483d6b0","Type":"ContainerStarted","Data":"171fa5816201748a5fdd69faa8fe6718668b13b93bdf337326a575ace87261de"} Jul 10 01:12:41.018928 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:41.018872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:41.100818 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:41.100783 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cs9gs"] Jul 10 01:12:41.101217 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:12:41.101190 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c77c9d_64a8_4008_aa90_49f7a2f99a79.slice/crio-8df1c2c0c67e800573792aa1e528b96c3f24dcdebb2899ab84c127572967cb5d WatchSource:0}: Error finding container 8df1c2c0c67e800573792aa1e528b96c3f24dcdebb2899ab84c127572967cb5d: Status 404 returned error can't find the container with id 8df1c2c0c67e800573792aa1e528b96c3f24dcdebb2899ab84c127572967cb5d Jul 10 01:12:41.169457 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:41.169425 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4cd8p"] Jul 10 01:12:41.171399 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:12:41.171367 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44cda98b_bb58_4d4b_a044_1b5501d1927b.slice/crio-745c10f8ae9b07888c7fee59f9fe3af197a84f1b511e96f0d555a4be3943bd02 WatchSource:0}: Error finding container 745c10f8ae9b07888c7fee59f9fe3af197a84f1b511e96f0d555a4be3943bd02: Status 404 returned error can't find the container with id 745c10f8ae9b07888c7fee59f9fe3af197a84f1b511e96f0d555a4be3943bd02 Jul 10 01:12:41.962100 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:41.962062 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" event={"ID":"83c77c9d-64a8-4008-aa90-49f7a2f99a79","Type":"ContainerStarted","Data":"8df1c2c0c67e800573792aa1e528b96c3f24dcdebb2899ab84c127572967cb5d"} Jul 10 01:12:41.963587 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:41.963541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" event={"ID":"44cda98b-bb58-4d4b-a044-1b5501d1927b","Type":"ContainerStarted","Data":"745c10f8ae9b07888c7fee59f9fe3af197a84f1b511e96f0d555a4be3943bd02"} Jul 10 01:12:42.970334 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.970236 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" event={"ID":"83c77c9d-64a8-4008-aa90-49f7a2f99a79","Type":"ContainerStarted","Data":"4753204e7b5f3ddabf9078782f1738302c1424804f922484d5b02d3e4401a812"} Jul 10 01:12:42.970334 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.970307 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:42.971938 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.971881 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" event={"ID":"1689f147-8298-4897-80a7-ccf30483d6b0","Type":"ContainerStarted","Data":"7c60c1d75bb64fdb74ef1a20ddab178c79d942ec69611feb95c2f269c1ae3daa"} Jul 10 01:12:42.972080 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.972008 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:42.973469 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.973437 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" event={"ID":"44cda98b-bb58-4d4b-a044-1b5501d1927b","Type":"ContainerStarted","Data":"a1b724b30717242df16a2cee80bef4ed83d77fcd4e6db118525d7857cb0f7f1e"} Jul 10 01:12:42.973581 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.973507 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:12:42.985266 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:42.985207 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" podStartSLOduration=1.398144885 podStartE2EDuration="2.985187981s" totalImagesPullingTime="1.587043096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:12:42.984259773 +0000 UTC m=+738.646054131" watchObservedRunningTime="2026-07-10 01:12:42.985187981 +0000 UTC m=+738.646982337" Jul 10 01:12:43.017806 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:43.017737 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" podStartSLOduration=1.5191828859999998 podStartE2EDuration="3.0177133s" totalImagesPullingTime="1.498530414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:12:43.017013226 +0000 UTC m=+738.678807581" watchObservedRunningTime="2026-07-10 01:12:43.0177133 +0000 UTC m=+738.679507655" Jul 10 01:12:43.019486 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:43.019434 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" podStartSLOduration=1.319715776 podStartE2EDuration="3.01941596s" totalImagesPullingTime="1.699700184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:12:43.003453976 +0000 UTC m=+738.665248332" watchObservedRunningTime="2026-07-10 01:12:43.01941596 +0000 UTC m=+738.681210314" Jul 10 01:12:48.984773 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:48.984735 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-db47c776-n5c2h" Jul 10 01:12:53.981767 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:53.981726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-cs9gs" Jul 10 01:12:53.982193 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:12:53.981875 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-4cd8p" Jul 10 01:13:00.730196 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.730151 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj"] Jul 10 01:13:00.733955 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.733928 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.737776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.737747 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 01:13:00.737949 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.737747 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 01:13:00.737949 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.737816 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xrfbf\"" Jul 10 01:13:00.737949 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.737833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 01:13:00.747108 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.747079 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj"] Jul 10 01:13:00.879779 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.879998 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.879998 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.879998 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879855 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsnn\" (UniqueName: \"kubernetes.io/projected/0e03befb-4213-4533-b9c2-819106e07e8a-kube-api-access-ftsnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.879998 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.879998 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.879932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e03befb-4213-4533-b9c2-819106e07e8a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981253 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981142 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981253 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981253 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981224 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981253 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981245 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsnn\" (UniqueName: \"kubernetes.io/projected/0e03befb-4213-4533-b9c2-819106e07e8a-kube-api-access-ftsnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981280 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e03befb-4213-4533-b9c2-819106e07e8a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981597 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981575 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981755 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981638 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.981755 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.981678 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.983685 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.983659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e03befb-4213-4533-b9c2-819106e07e8a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.983946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.983930 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e03befb-4213-4533-b9c2-819106e07e8a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:00.990980 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:00.990956 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsnn\" (UniqueName: \"kubernetes.io/projected/0e03befb-4213-4533-b9c2-819106e07e8a-kube-api-access-ftsnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj\" (UID: \"0e03befb-4213-4533-b9c2-819106e07e8a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:01.046423 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:01.046377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:01.211730 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:01.211702 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj"] Jul 10 01:13:01.213542 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:13:01.213512 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e03befb_4213_4533_b9c2_819106e07e8a.slice/crio-e6c3e7dd3e4933993f549d0685067b8b267f10c7653900a101c6647de49ccedd WatchSource:0}: Error finding container e6c3e7dd3e4933993f549d0685067b8b267f10c7653900a101c6647de49ccedd: Status 404 returned error can't find the container with id e6c3e7dd3e4933993f549d0685067b8b267f10c7653900a101c6647de49ccedd Jul 10 01:13:02.074432 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:02.074373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" event={"ID":"0e03befb-4213-4533-b9c2-819106e07e8a","Type":"ContainerStarted","Data":"e6c3e7dd3e4933993f549d0685067b8b267f10c7653900a101c6647de49ccedd"} Jul 10 01:13:03.081390 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:03.081349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" event={"ID":"0e03befb-4213-4533-b9c2-819106e07e8a","Type":"ContainerStarted","Data":"06ae22988b0fa5cfec95bddc03a5067680c0148ae88461704460a8b66c14aaae"} Jul 10 01:13:03.098644 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:03.098579 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" podStartSLOduration=1.95272222 podStartE2EDuration="3.09856048s" totalImagesPullingTime="1.14583826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:13:03.096700957 +0000 UTC m=+758.758495311" watchObservedRunningTime="2026-07-10 01:13:03.09856048 +0000 UTC m=+758.760354838" Jul 10 01:13:11.046776 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:11.046728 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:11.047327 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:11.046816 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:11.059550 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:11.059516 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:11.132582 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:11.132547 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj" Jul 10 01:13:29.036291 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.036253 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb"] Jul 10 01:13:29.040171 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.040146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.042846 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.042810 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 10 01:13:29.049403 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.049372 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb"] Jul 10 01:13:29.130081 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/925b157a-59ac-4b69-af7d-3d55598d9f5e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.130281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130121 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvpzj\" (UniqueName: \"kubernetes.io/projected/925b157a-59ac-4b69-af7d-3d55598d9f5e-kube-api-access-qvpzj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.130281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.130281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.130396 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130323 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.130396 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.130355 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231013 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.230972 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/925b157a-59ac-4b69-af7d-3d55598d9f5e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231046 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvpzj\" (UniqueName: \"kubernetes.io/projected/925b157a-59ac-4b69-af7d-3d55598d9f5e-kube-api-access-qvpzj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231130 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231222 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231649 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231610 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.231649 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.231628 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.232064 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.232041 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.233534 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.233517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/925b157a-59ac-4b69-af7d-3d55598d9f5e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.233645 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.233628 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/925b157a-59ac-4b69-af7d-3d55598d9f5e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.238859 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.238831 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvpzj\" (UniqueName: \"kubernetes.io/projected/925b157a-59ac-4b69-af7d-3d55598d9f5e-kube-api-access-qvpzj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l6hfb\" (UID: \"925b157a-59ac-4b69-af7d-3d55598d9f5e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.351838 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.351792 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:29.496016 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:29.495982 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb"] Jul 10 01:13:29.497977 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:13:29.497944 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925b157a_59ac_4b69_af7d_3d55598d9f5e.slice/crio-d120d4b5d0a8d1f1506f8d49e8911475313aaf8a901bc8405a5cf44be8eaf1a3 WatchSource:0}: Error finding container d120d4b5d0a8d1f1506f8d49e8911475313aaf8a901bc8405a5cf44be8eaf1a3: Status 404 returned error can't find the container with id d120d4b5d0a8d1f1506f8d49e8911475313aaf8a901bc8405a5cf44be8eaf1a3 Jul 10 01:13:30.214109 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:30.214068 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" event={"ID":"925b157a-59ac-4b69-af7d-3d55598d9f5e","Type":"ContainerStarted","Data":"29992c3788ea634dd635a2ae001bf0237801aa85ce71b4c60b020591e4c5782a"} Jul 10 01:13:30.214544 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:30.214127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" event={"ID":"925b157a-59ac-4b69-af7d-3d55598d9f5e","Type":"ContainerStarted","Data":"d120d4b5d0a8d1f1506f8d49e8911475313aaf8a901bc8405a5cf44be8eaf1a3"} Jul 10 01:13:30.231219 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:30.231165 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" podStartSLOduration=1.048623352 podStartE2EDuration="1.231146116s" totalImagesPullingTime="182.522764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:13:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:13:30.229539083 +0000 UTC m=+785.891333441" watchObservedRunningTime="2026-07-10 01:13:30.231146116 +0000 UTC m=+785.892940471" Jul 10 01:13:35.344025 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.343979 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc"] Jul 10 01:13:35.347633 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.347600 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.350355 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.350328 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 01:13:35.359828 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.359792 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc"] Jul 10 01:13:35.391145 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.391145 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcxj\" (UniqueName: \"kubernetes.io/projected/124f314c-194d-43ab-95e2-b048fa6a7d3f-kube-api-access-hvcxj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.391383 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391219 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.391383 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124f314c-194d-43ab-95e2-b048fa6a7d3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.391383 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.391383 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.391291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.491996 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.491955 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcxj\" (UniqueName: \"kubernetes.io/projected/124f314c-194d-43ab-95e2-b048fa6a7d3f-kube-api-access-hvcxj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124f314c-194d-43ab-95e2-b048fa6a7d3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492194 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492561 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492534 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492647 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.492647 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.492628 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.494684 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.494661 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124f314c-194d-43ab-95e2-b048fa6a7d3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.494976 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.494960 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124f314c-194d-43ab-95e2-b048fa6a7d3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.501255 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.501227 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcxj\" (UniqueName: \"kubernetes.io/projected/124f314c-194d-43ab-95e2-b048fa6a7d3f-kube-api-access-hvcxj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc\" (UID: \"124f314c-194d-43ab-95e2-b048fa6a7d3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.661214 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.661121 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:35.800008 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:35.799975 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc"] Jul 10 01:13:35.801992 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:13:35.801963 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124f314c_194d_43ab_95e2_b048fa6a7d3f.slice/crio-a56238f801dc0b49fdee954062bdc9e61226f62b5afffd1f984ea10f62d3cd3c WatchSource:0}: Error finding container a56238f801dc0b49fdee954062bdc9e61226f62b5afffd1f984ea10f62d3cd3c: Status 404 returned error can't find the container with id a56238f801dc0b49fdee954062bdc9e61226f62b5afffd1f984ea10f62d3cd3c Jul 10 01:13:36.245687 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:36.245592 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" event={"ID":"124f314c-194d-43ab-95e2-b048fa6a7d3f","Type":"ContainerStarted","Data":"e628833cee7a17584b1277416653c97dbf22852e195a9e724be1d55d32140a65"} Jul 10 01:13:36.245844 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:36.245691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" event={"ID":"124f314c-194d-43ab-95e2-b048fa6a7d3f","Type":"ContainerStarted","Data":"a56238f801dc0b49fdee954062bdc9e61226f62b5afffd1f984ea10f62d3cd3c"} Jul 10 01:13:36.262673 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:36.262616 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" podStartSLOduration=1.100462356 podStartE2EDuration="1.262599912s" totalImagesPullingTime="162.137556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:13:36.261195106 +0000 UTC m=+791.922989461" watchObservedRunningTime="2026-07-10 01:13:36.262599912 +0000 UTC m=+791.924394264" Jul 10 01:13:39.352743 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:39.352700 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:39.353245 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:39.352770 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:39.365616 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:39.365578 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:40.280621 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:40.280592 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l6hfb" Jul 10 01:13:45.661603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:45.661508 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:45.661603 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:45.661589 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:45.674404 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:45.674370 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:13:46.310778 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:13:46.310745 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc" Jul 10 01:15:00.140233 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.140194 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:15:00.142705 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.142682 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:15:00.153147 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.153118 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:15:00.245234 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.245192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch4gp\" (UniqueName: \"kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp\") pod \"maas-api-key-cleanup-29727435-jzsqq\" (UID: \"daac6260-ed9d-40c8-b716-d45d1c13a3fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:15:00.345882 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.345840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch4gp\" (UniqueName: \"kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp\") pod \"maas-api-key-cleanup-29727435-jzsqq\" (UID: \"daac6260-ed9d-40c8-b716-d45d1c13a3fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:15:00.354145 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.354112 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch4gp\" (UniqueName: \"kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp\") pod \"maas-api-key-cleanup-29727435-jzsqq\" (UID: \"daac6260-ed9d-40c8-b716-d45d1c13a3fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:15:00.454136 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.454043 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:15:00.595884 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.594838 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:15:00.598297 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:15:00.598262 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaac6260_ed9d_40c8_b716_d45d1c13a3fc.slice/crio-5ddb124397c3a9f7cede69823d4284f5584ecffb6cc8cd788a2d99cbdb35eec6 WatchSource:0}: Error finding container 5ddb124397c3a9f7cede69823d4284f5584ecffb6cc8cd788a2d99cbdb35eec6: Status 404 returned error can't find the container with id 5ddb124397c3a9f7cede69823d4284f5584ecffb6cc8cd788a2d99cbdb35eec6 Jul 10 01:15:00.681066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:00.681025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" event={"ID":"daac6260-ed9d-40c8-b716-d45d1c13a3fc","Type":"ContainerStarted","Data":"5ddb124397c3a9f7cede69823d4284f5584ecffb6cc8cd788a2d99cbdb35eec6"} Jul 10 01:15:03.698695 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:03.698647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" event={"ID":"daac6260-ed9d-40c8-b716-d45d1c13a3fc","Type":"ContainerStarted","Data":"ad6c8e2a79cc7b7a22165a9d3bd8ac9df297a0e8c354a777a1f1fbf39e8a8b37"} Jul 10 01:15:03.715006 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:03.714947 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" podStartSLOduration=1.039698155 podStartE2EDuration="3.714926507s" totalImagesPullingTime="2.675228352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:15:03.71357751 +0000 UTC m=+879.375371868" watchObservedRunningTime="2026-07-10 01:15:03.714926507 +0000 UTC m=+879.376720862" Jul 10 01:15:24.872236 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:24.872209 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:15:24.872770 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:15:24.872728 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:16:00.011239 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:00.011200 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:16:00.011781 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:00.011410 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" podUID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" containerName="cleanup" containerID="cri-o://ad6c8e2a79cc7b7a22165a9d3bd8ac9df297a0e8c354a777a1f1fbf39e8a8b37" gracePeriod=30 Jul 10 01:16:30.147777 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.147731 2565 generic.go:356] "Generic (PLEG): container finished" podID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" containerID="ad6c8e2a79cc7b7a22165a9d3bd8ac9df297a0e8c354a777a1f1fbf39e8a8b37" exitCode=137 Jul 10 01:16:30.148167 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.147824 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" event={"ID":"daac6260-ed9d-40c8-b716-d45d1c13a3fc","Type":"ContainerDied","Data":"ad6c8e2a79cc7b7a22165a9d3bd8ac9df297a0e8c354a777a1f1fbf39e8a8b37"} Jul 10 01:16:30.205450 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.205419 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:16:30.362040 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.362002 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch4gp\" (UniqueName: \"kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp\") pod \"daac6260-ed9d-40c8-b716-d45d1c13a3fc\" (UID: \"daac6260-ed9d-40c8-b716-d45d1c13a3fc\") " Jul 10 01:16:30.364368 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.364330 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp" (OuterVolumeSpecName: "kube-api-access-ch4gp") pod "daac6260-ed9d-40c8-b716-d45d1c13a3fc" (UID: "daac6260-ed9d-40c8-b716-d45d1c13a3fc"). InnerVolumeSpecName "kube-api-access-ch4gp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:16:30.463326 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:30.463280 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch4gp\" (UniqueName: \"kubernetes.io/projected/daac6260-ed9d-40c8-b716-d45d1c13a3fc-kube-api-access-ch4gp\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:16:31.154800 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:31.154762 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" event={"ID":"daac6260-ed9d-40c8-b716-d45d1c13a3fc","Type":"ContainerDied","Data":"5ddb124397c3a9f7cede69823d4284f5584ecffb6cc8cd788a2d99cbdb35eec6"} Jul 10 01:16:31.154800 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:31.154792 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq" Jul 10 01:16:31.155390 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:31.154839 2565 scope.go:117] "RemoveContainer" containerID="ad6c8e2a79cc7b7a22165a9d3bd8ac9df297a0e8c354a777a1f1fbf39e8a8b37" Jul 10 01:16:31.171852 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:31.171815 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:16:31.177578 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:31.177547 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727435-jzsqq"] Jul 10 01:16:32.943358 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:16:32.943326 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" path="/var/lib/kubelet/pods/daac6260-ed9d-40c8-b716-d45d1c13a3fc/volumes" Jul 10 01:18:13.634542 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:13.634505 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:18:13.635149 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:13.634766 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7867f4b576-w8mhc" podUID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" containerName="manager" containerID="cri-o://9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50" gracePeriod=30 Jul 10 01:18:13.903687 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:13.903654 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:18:14.071180 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.071137 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert\") pod \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " Jul 10 01:18:14.071388 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.071202 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rkns\" (UniqueName: \"kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns\") pod \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " Jul 10 01:18:14.071388 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.071257 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca\") pod \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\" (UID: \"18a3c9f7-33ba-4d9f-95ff-0797e719adf0\") " Jul 10 01:18:14.071621 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.071597 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca" (OuterVolumeSpecName: "service-ca") pod "18a3c9f7-33ba-4d9f-95ff-0797e719adf0" (UID: "18a3c9f7-33ba-4d9f-95ff-0797e719adf0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:18:14.073562 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.073523 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "18a3c9f7-33ba-4d9f-95ff-0797e719adf0" (UID: "18a3c9f7-33ba-4d9f-95ff-0797e719adf0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:18:14.073562 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.073552 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns" (OuterVolumeSpecName: "kube-api-access-6rkns") pod "18a3c9f7-33ba-4d9f-95ff-0797e719adf0" (UID: "18a3c9f7-33ba-4d9f-95ff-0797e719adf0"). InnerVolumeSpecName "kube-api-access-6rkns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:18:14.172232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.172132 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-webhook-cert\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:18:14.172232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.172175 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rkns\" (UniqueName: \"kubernetes.io/projected/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-kube-api-access-6rkns\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:18:14.172232 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.172187 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a3c9f7-33ba-4d9f-95ff-0797e719adf0-service-ca\") on node \"ip-10-0-128-136.ec2.internal\" DevicePath \"\"" Jul 10 01:18:14.701326 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.701287 2565 generic.go:356] "Generic (PLEG): container finished" podID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" containerID="9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50" exitCode=0 Jul 10 01:18:14.701768 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.701333 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-w8mhc" Jul 10 01:18:14.701768 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.701370 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-w8mhc" event={"ID":"18a3c9f7-33ba-4d9f-95ff-0797e719adf0","Type":"ContainerDied","Data":"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50"} Jul 10 01:18:14.701768 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.701427 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-w8mhc" event={"ID":"18a3c9f7-33ba-4d9f-95ff-0797e719adf0","Type":"ContainerDied","Data":"c447271d024892a85dde279359e1df08d28ef01070bb6af4798e24bed32f6439"} Jul 10 01:18:14.701768 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.701449 2565 scope.go:117] "RemoveContainer" containerID="9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50" Jul 10 01:18:14.711397 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.711376 2565 scope.go:117] "RemoveContainer" containerID="9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50" Jul 10 01:18:14.711691 ip-10-0-128-136 kubenswrapper[2565]: E0710 01:18:14.711675 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50\": container with ID starting with 9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50 not found: ID does not exist" containerID="9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50" Jul 10 01:18:14.711740 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.711700 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50"} err="failed to get container status \"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50\": rpc error: code = NotFound desc = could not find container \"9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50\": container with ID starting with 9820470f02387a039b0586bc73fad9886870cdbf877d37f20b29c8120d063e50 not found: ID does not exist" Jul 10 01:18:14.721984 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.721949 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:18:14.723903 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.723869 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7867f4b576-w8mhc"] Jul 10 01:18:14.943421 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:14.943392 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" path="/var/lib/kubelet/pods/18a3c9f7-33ba-4d9f-95ff-0797e719adf0/volumes" Jul 10 01:18:15.525388 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525352 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7867f4b576-k4rpb"] Jul 10 01:18:15.525741 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525728 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" containerName="cleanup" Jul 10 01:18:15.525788 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525743 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" containerName="cleanup" Jul 10 01:18:15.525788 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525753 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" containerName="manager" Jul 10 01:18:15.525788 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525759 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" containerName="manager" Jul 10 01:18:15.525878 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525811 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="daac6260-ed9d-40c8-b716-d45d1c13a3fc" containerName="cleanup" Jul 10 01:18:15.525878 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.525821 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a3c9f7-33ba-4d9f-95ff-0797e719adf0" containerName="manager" Jul 10 01:18:15.530247 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.530228 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.534267 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.534237 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 01:18:15.534445 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.534300 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 01:18:15.534548 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.534237 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q7hpk\"" Jul 10 01:18:15.537821 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.537695 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7867f4b576-k4rpb"] Jul 10 01:18:15.683946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.683865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db9577b5-5ac9-4a6c-8338-2d6a73947640-service-ca\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.684151 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.683990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db9577b5-5ac9-4a6c-8338-2d6a73947640-webhook-cert\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.684151 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.684069 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7m9\" (UniqueName: \"kubernetes.io/projected/db9577b5-5ac9-4a6c-8338-2d6a73947640-kube-api-access-ht7m9\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.785144 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.785050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7m9\" (UniqueName: \"kubernetes.io/projected/db9577b5-5ac9-4a6c-8338-2d6a73947640-kube-api-access-ht7m9\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.785144 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.785119 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db9577b5-5ac9-4a6c-8338-2d6a73947640-service-ca\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.785548 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.785160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db9577b5-5ac9-4a6c-8338-2d6a73947640-webhook-cert\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.785947 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.785924 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db9577b5-5ac9-4a6c-8338-2d6a73947640-service-ca\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.787823 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.787801 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db9577b5-5ac9-4a6c-8338-2d6a73947640-webhook-cert\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.793758 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.793729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7m9\" (UniqueName: \"kubernetes.io/projected/db9577b5-5ac9-4a6c-8338-2d6a73947640-kube-api-access-ht7m9\") pod \"maas-controller-7867f4b576-k4rpb\" (UID: \"db9577b5-5ac9-4a6c-8338-2d6a73947640\") " pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.841568 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.841532 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:15.979689 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.979659 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7867f4b576-k4rpb"] Jul 10 01:18:15.980811 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:18:15.980782 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9577b5_5ac9_4a6c_8338_2d6a73947640.slice/crio-719e35a234e645d4b1a0c912db1c833b5fb4012274d805ab2bd6f90bb5f53f49 WatchSource:0}: Error finding container 719e35a234e645d4b1a0c912db1c833b5fb4012274d805ab2bd6f90bb5f53f49: Status 404 returned error can't find the container with id 719e35a234e645d4b1a0c912db1c833b5fb4012274d805ab2bd6f90bb5f53f49 Jul 10 01:18:15.982220 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:15.982197 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 01:18:16.716024 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:16.715928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-k4rpb" event={"ID":"db9577b5-5ac9-4a6c-8338-2d6a73947640","Type":"ContainerStarted","Data":"ce5b5e378e5ebf89edc886b6f7914e7ee603954f3779d13d1d869fc3782cc540"} Jul 10 01:18:16.716024 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:16.715986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7867f4b576-k4rpb" event={"ID":"db9577b5-5ac9-4a6c-8338-2d6a73947640","Type":"ContainerStarted","Data":"719e35a234e645d4b1a0c912db1c833b5fb4012274d805ab2bd6f90bb5f53f49"} Jul 10 01:18:16.716024 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:16.716015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:18:16.733403 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:16.733349 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7867f4b576-k4rpb" podStartSLOduration=1.338412203 podStartE2EDuration="1.733331881s" totalImagesPullingTime="394.919678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:18:16.730435036 +0000 UTC m=+1072.392229388" watchObservedRunningTime="2026-07-10 01:18:16.733331881 +0000 UTC m=+1072.395126235" Jul 10 01:18:27.726141 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:18:27.726108 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7867f4b576-k4rpb" Jul 10 01:20:24.908835 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:20:24.908807 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:20:24.909318 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:20:24.909298 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:25:24.942328 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:25:24.942294 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:25:24.944757 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:25:24.944736 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:26:57.953741 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:57.953704 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dtb2r_c957deb5-7765-4339-860b-41f7b071d17c/manager/0.log" Jul 10 01:26:58.263865 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:58.263767 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7867f4b576-k4rpb_db9577b5-5ac9-4a6c-8338-2d6a73947640/manager/0.log" Jul 10 01:26:58.393049 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:58.393008 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-b96gk_8486740b-c1e4-4264-ba54-ee7b7cba2806/server/0.log" Jul 10 01:26:58.528752 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:58.528662 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hqzx2_6d714dfa-a9d0-4f33-9916-03246b7769fe/manager/0.log" Jul 10 01:26:58.786085 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:58.786012 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c74cbdcf-tstbq_c99fb414-fef1-4d3b-a091-4159cde2fc00/copy-manifests/0.log" Jul 10 01:26:58.799242 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:58.799212 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c74cbdcf-tstbq_c99fb414-fef1-4d3b-a091-4159cde2fc00/manager/0.log" Jul 10 01:26:59.337133 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:59.337105 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-db47c776-n5c2h_1689f147-8298-4897-80a7-ccf30483d6b0/maas-api/0.log" Jul 10 01:26:59.460013 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:26:59.459985 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-j6tbj_f510aa4a-df1f-42b2-b75e-36b25cdd538f/postgres/0.log" Jul 10 01:27:00.317454 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.317417 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/util/0.log" Jul 10 01:27:00.324050 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.324024 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/pull/0.log" Jul 10 01:27:00.330611 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.330565 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/extract/0.log" Jul 10 01:27:00.448863 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.448827 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/pull/0.log" Jul 10 01:27:00.454425 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.454406 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/extract/0.log" Jul 10 01:27:00.459488 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.459464 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/util/0.log" Jul 10 01:27:00.585013 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.584991 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/util/0.log" Jul 10 01:27:00.591143 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.591121 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/pull/0.log" Jul 10 01:27:00.597878 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.597855 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/extract/0.log" Jul 10 01:27:00.738540 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.738507 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/util/0.log" Jul 10 01:27:00.744676 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.744656 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/pull/0.log" Jul 10 01:27:00.750760 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:00.750738 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/extract/0.log" Jul 10 01:27:01.018714 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:01.018628 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9wncj_1ca5ccdb-77bc-410a-93c8-513f2e133274/manager/0.log" Jul 10 01:27:01.402071 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:01.402037 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dszz9_267daf1f-9dd8-47d7-885e-a687f2ca1710/registry-server/0.log" Jul 10 01:27:02.357787 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:02.357761 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c88x4_e8e60633-cdd4-449a-9d40-de71e4fd5702/discovery/0.log" Jul 10 01:27:02.633379 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:02.633301 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd866567d-p5rjl_b62948b9-ed2a-488c-80e4-31aa5d73bddb/kube-auth-proxy/0.log" Jul 10 01:27:02.895692 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:02.895614 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4cd8p_44cda98b-bb58-4d4b-a044-1b5501d1927b/payload-pre-processing/0.log" Jul 10 01:27:03.023908 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:03.023856 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cs9gs_83c77c9d-64a8-4008-aa90-49f7a2f99a79/payload-processing/0.log" Jul 10 01:27:03.530569 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:03.530535 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-z62qj_0e03befb-4213-4533-b9c2-819106e07e8a/main/0.log" Jul 10 01:27:03.796098 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:03.796018 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-l6hfb_925b157a-59ac-4b69-af7d-3d55598d9f5e/main/0.log" Jul 10 01:27:03.916071 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:03.916033 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsr6bc_124f314c-194d-43ab-95e2-b048fa6a7d3f/main/0.log" Jul 10 01:27:11.680277 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:11.680237 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x5cp6_389ecb05-4601-4cc5-a414-04cca57b42c0/global-pull-secret-syncer/0.log" Jul 10 01:27:11.750017 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:11.749989 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l7v52_91de7b74-be93-4945-9d06-c036c40e4f1b/konnectivity-agent/0.log" Jul 10 01:27:11.793124 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:11.793095 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-136.ec2.internal_78350d0908fbb8795cb0dc5d10a96325/haproxy/0.log" Jul 10 01:27:16.436174 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.436067 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/extract/0.log" Jul 10 01:27:16.463946 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.463909 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/util/0.log" Jul 10 01:27:16.484680 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.484627 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tvl9k_5000a8a0-1351-4d2e-b4c1-510c56135580/pull/0.log" Jul 10 01:27:16.510383 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.510344 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/extract/0.log" Jul 10 01:27:16.530530 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.530500 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/util/0.log" Jul 10 01:27:16.550617 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.550586 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d4wb7_b47a2b76-4b9a-4f85-bf40-91142ea99bfa/pull/0.log" Jul 10 01:27:16.575423 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.575391 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/extract/0.log" Jul 10 01:27:16.596380 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.596349 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/util/0.log" Jul 10 01:27:16.617138 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.617109 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hql8c_bdb583c9-2562-4ab8-bd86-335e926855f5/pull/0.log" Jul 10 01:27:16.651693 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.651666 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/extract/0.log" Jul 10 01:27:16.676318 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.676290 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/util/0.log" Jul 10 01:27:16.700401 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.700329 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1759ds_b772923b-4b26-43ca-9596-deb8631bdeae/pull/0.log" Jul 10 01:27:16.860143 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.860113 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9wncj_1ca5ccdb-77bc-410a-93c8-513f2e133274/manager/0.log" Jul 10 01:27:16.969794 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:16.969714 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dszz9_267daf1f-9dd8-47d7-885e-a687f2ca1710/registry-server/0.log" Jul 10 01:27:19.367772 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.367745 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8ngn_a9b45868-72c9-4fef-b531-f1913279583a/node-exporter/0.log" Jul 10 01:27:19.392096 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.392073 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8ngn_a9b45868-72c9-4fef-b531-f1913279583a/kube-rbac-proxy/0.log" Jul 10 01:27:19.414545 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.414519 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8ngn_a9b45868-72c9-4fef-b531-f1913279583a/init-textfile/0.log" Jul 10 01:27:19.847268 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.847241 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d5bb68b45-g5n2z_34b386a4-8a51-4a7b-91b4-0f0640f1783d/telemeter-client/0.log" Jul 10 01:27:19.874572 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.874546 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d5bb68b45-g5n2z_34b386a4-8a51-4a7b-91b4-0f0640f1783d/reload/0.log" Jul 10 01:27:19.898471 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:19.898442 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d5bb68b45-g5n2z_34b386a4-8a51-4a7b-91b4-0f0640f1783d/kube-rbac-proxy/0.log" Jul 10 01:27:22.048110 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.048069 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd"] Jul 10 01:27:22.052118 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.052091 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.054705 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.054678 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jcgw7\"/\"kube-root-ca.crt\"" Jul 10 01:27:22.055680 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.055659 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jcgw7\"/\"openshift-service-ca.crt\"" Jul 10 01:27:22.055797 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.055697 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jcgw7\"/\"default-dockercfg-sk8zq\"" Jul 10 01:27:22.060045 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.059758 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd"] Jul 10 01:27:22.133339 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.133293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-sys\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.133541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.133369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-lib-modules\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.133541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.133403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-proc\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.133541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.133438 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-podres\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.133541 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.133476 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2zzb\" (UniqueName: \"kubernetes.io/projected/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-kube-api-access-d2zzb\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.234850 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-sys\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-lib-modules\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-proc\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-podres\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234967 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-sys\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.234973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2zzb\" (UniqueName: \"kubernetes.io/projected/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-kube-api-access-d2zzb\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235061 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.235047 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-proc\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.235113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-podres\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.235282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.235117 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-lib-modules\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.243366 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.243338 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2zzb\" (UniqueName: \"kubernetes.io/projected/46fe6d42-fbed-4c4b-97d9-0e1be1bdc787-kube-api-access-d2zzb\") pod \"perf-node-gather-daemonset-h8fdd\" (UID: \"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787\") " pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.363913 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.363851 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:22.508276 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:22.508244 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd"] Jul 10 01:27:22.510288 ip-10-0-128-136 kubenswrapper[2565]: W0710 01:27:22.510256 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod46fe6d42_fbed_4c4b_97d9_0e1be1bdc787.slice/crio-1aa8d8518e6f01496691301c343ad97d36c2050c51d88a000eecf816fa492e43 WatchSource:0}: Error finding container 1aa8d8518e6f01496691301c343ad97d36c2050c51d88a000eecf816fa492e43: Status 404 returned error can't find the container with id 1aa8d8518e6f01496691301c343ad97d36c2050c51d88a000eecf816fa492e43 Jul 10 01:27:23.495729 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:23.495693 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" event={"ID":"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787","Type":"ContainerStarted","Data":"44bd1f27fc3dda52be8b75fd3fbe209e580f3d121a1b3c4789619492ae6eab99"} Jul 10 01:27:23.496173 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:23.495747 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" event={"ID":"46fe6d42-fbed-4c4b-97d9-0e1be1bdc787","Type":"ContainerStarted","Data":"1aa8d8518e6f01496691301c343ad97d36c2050c51d88a000eecf816fa492e43"} Jul 10 01:27:23.496173 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:23.495906 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:23.512078 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:23.512023 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" podStartSLOduration=1.512005818 podStartE2EDuration="1.512005818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:27:23.509310777 +0000 UTC m=+1619.171105132" watchObservedRunningTime="2026-07-10 01:27:23.512005818 +0000 UTC m=+1619.173800173" Jul 10 01:27:23.992281 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:23.992254 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvklj_e1e570c0-1149-4943-bffc-5d2f0f8c216d/dns/0.log" Jul 10 01:27:24.013136 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:24.013105 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvklj_e1e570c0-1149-4943-bffc-5d2f0f8c216d/kube-rbac-proxy/0.log" Jul 10 01:27:24.060824 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:24.060797 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q9pks_1833f960-875b-4907-90c8-bcdb3c5ffb41/dns-node-resolver/0.log" Jul 10 01:27:24.600962 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:24.600929 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b46fb8c9d-x4l6d_30c4f995-51cf-4b5e-adcc-2819afcfb7b6/registry/0.log" Jul 10 01:27:24.621101 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:24.621072 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5kcqk_894b8144-59f5-4387-b59b-4564ea772cf7/node-ca/0.log" Jul 10 01:27:25.610856 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:25.610823 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c88x4_e8e60633-cdd4-449a-9d40-de71e4fd5702/discovery/0.log" Jul 10 01:27:25.657286 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:25.657261 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd866567d-p5rjl_b62948b9-ed2a-488c-80e4-31aa5d73bddb/kube-auth-proxy/0.log" Jul 10 01:27:25.723583 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:25.723553 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4cd8p_44cda98b-bb58-4d4b-a044-1b5501d1927b/payload-pre-processing/0.log" Jul 10 01:27:25.745901 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:25.745857 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cs9gs_83c77c9d-64a8-4008-aa90-49f7a2f99a79/payload-processing/0.log" Jul 10 01:27:26.425002 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:26.424971 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbxdj_6ab167a5-1592-4033-ba8e-d9c05b9d17d1/serve-healthcheck-canary/0.log" Jul 10 01:27:27.138495 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:27.138461 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpsz6_fab23a67-fd7e-4676-a278-0bb5eb91987c/kube-rbac-proxy/0.log" Jul 10 01:27:27.181142 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:27.181114 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpsz6_fab23a67-fd7e-4676-a278-0bb5eb91987c/exporter/0.log" Jul 10 01:27:27.216465 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:27.216429 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpsz6_fab23a67-fd7e-4676-a278-0bb5eb91987c/extractor/0.log" Jul 10 01:27:29.364713 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.364677 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dtb2r_c957deb5-7765-4339-860b-41f7b071d17c/manager/0.log" Jul 10 01:27:29.441500 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.441465 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7867f4b576-k4rpb_db9577b5-5ac9-4a6c-8338-2d6a73947640/manager/0.log" Jul 10 01:27:29.461008 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.460979 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-b96gk_8486740b-c1e4-4264-ba54-ee7b7cba2806/server/0.log" Jul 10 01:27:29.510486 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.510458 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hqzx2_6d714dfa-a9d0-4f33-9916-03246b7769fe/manager/0.log" Jul 10 01:27:29.512636 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.512617 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jcgw7/perf-node-gather-daemonset-h8fdd" Jul 10 01:27:29.608334 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.608285 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c74cbdcf-tstbq_c99fb414-fef1-4d3b-a091-4159cde2fc00/manager/0.log" Jul 10 01:27:29.624389 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:29.624310 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c74cbdcf-tstbq_c99fb414-fef1-4d3b-a091-4159cde2fc00/copy-manifests/0.log" Jul 10 01:27:32.426941 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:32.426900 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c88x4_e8e60633-cdd4-449a-9d40-de71e4fd5702/discovery/0.log" Jul 10 01:27:32.451411 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:32.451382 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd866567d-p5rjl_b62948b9-ed2a-488c-80e4-31aa5d73bddb/kube-auth-proxy/0.log" Jul 10 01:27:32.509294 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:32.509262 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4cd8p_44cda98b-bb58-4d4b-a044-1b5501d1927b/payload-pre-processing/0.log" Jul 10 01:27:32.523292 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:32.523264 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cs9gs_83c77c9d-64a8-4008-aa90-49f7a2f99a79/payload-processing/0.log" Jul 10 01:27:33.732625 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:33.732516 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-c88x4_e8e60633-cdd4-449a-9d40-de71e4fd5702/discovery/0.log" Jul 10 01:27:33.766856 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:33.766828 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd866567d-p5rjl_b62948b9-ed2a-488c-80e4-31aa5d73bddb/kube-auth-proxy/0.log" Jul 10 01:27:33.817134 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:33.817102 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4cd8p_44cda98b-bb58-4d4b-a044-1b5501d1927b/payload-pre-processing/0.log" Jul 10 01:27:33.833066 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:33.833020 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cs9gs_83c77c9d-64a8-4008-aa90-49f7a2f99a79/payload-processing/0.log" Jul 10 01:27:37.907786 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:37.907757 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/kube-multus-additional-cni-plugins/0.log" Jul 10 01:27:37.928413 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:37.928390 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/egress-router-binary-copy/0.log" Jul 10 01:27:37.948508 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:37.948480 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/cni-plugins/0.log" Jul 10 01:27:37.969794 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:37.969767 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/bond-cni-plugin/0.log" Jul 10 01:27:37.990960 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:37.990928 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/routeoverride-cni/0.log" Jul 10 01:27:38.011320 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:38.011293 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/whereabouts-cni-bincopy/0.log" Jul 10 01:27:38.036040 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:38.036012 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8frj_1e5f96c9-8b03-4bba-a377-5d43f5d90c50/whereabouts-cni/0.log" Jul 10 01:27:38.069019 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:38.068965 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f9kxd_5227952a-01f8-411a-870b-b208ca062c27/kube-multus/0.log" Jul 10 01:27:38.197282 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:38.197170 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rztvp_cf888257-f4f1-47c2-a54b-eba014c092b6/network-metrics-daemon/0.log" Jul 10 01:27:38.215262 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:38.215232 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rztvp_cf888257-f4f1-47c2-a54b-eba014c092b6/kube-rbac-proxy/0.log" Jul 10 01:27:39.173489 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.173458 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-controller/0.log" Jul 10 01:27:39.190928 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.190861 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/1.log" Jul 10 01:27:39.191101 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.190946 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovn-acl-logging/0.log" Jul 10 01:27:39.210831 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.210799 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/kube-rbac-proxy-node/0.log" Jul 10 01:27:39.231312 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.231278 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/kube-rbac-proxy-ovn-metrics/0.log" Jul 10 01:27:39.248530 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.248502 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/northd/0.log" Jul 10 01:27:39.268749 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.268712 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/nbdb/0.log" Jul 10 01:27:39.292925 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.292871 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/sbdb/0.log" Jul 10 01:27:39.334145 ip-10-0-128-136 kubenswrapper[2565]: I0710 01:27:39.334110 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqgp9_bafc5e2d-4876-4e8a-9c9d-740a854045c2/ovnkube-controller/0.log"