Jul 02 22:02:27.806984 ip-10-0-128-202 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 22:02:27.806996 ip-10-0-128-202 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 22:02:27.807003 ip-10-0-128-202 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 22:02:27.807245 ip-10-0-128-202 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 22:02:37.807780 ip-10-0-128-202 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 22:02:37.807795 ip-10-0-128-202 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d9f2b842fa764630b55bfd9aa51432c0 -- Jul 02 22:04:14.892628 ip-10-0-128-202 systemd[1]: Starting Kubernetes Kubelet... Jul 02 22:04:15.316969 ip-10-0-128-202 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 22:04:15.316969 ip-10-0-128-202 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 22:04:15.316969 ip-10-0-128-202 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 22:04:15.316969 ip-10-0-128-202 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 22:04:15.316969 ip-10-0-128-202 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 22:04:15.318154 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.318027 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 22:04:15.321585 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321569 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.321585 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321584 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321588 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321592 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321595 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321598 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321601 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321604 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321606 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321609 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321612 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321615 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321618 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321620 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321623 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321626 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321628 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321631 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321634 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321637 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321640 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.321656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321643 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321646 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321648 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321651 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321654 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321656 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321659 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321662 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321665 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321668 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321671 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321674 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321676 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321680 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321683 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321686 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321689 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321692 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321694 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321697 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.322134 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321699 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321702 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321704 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321709 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321712 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321716 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321718 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321721 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321724 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321727 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321729 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321731 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321734 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321737 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321741 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321743 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321746 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321749 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321751 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.322633 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321753 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321756 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321758 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321761 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321764 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321766 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321769 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321771 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321774 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321776 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321779 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321783 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321786 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321790 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321793 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321796 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321799 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321801 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321804 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.323105 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321807 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321809 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321812 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321814 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321818 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321820 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321823 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321825 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321828 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321832 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321835 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321838 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321841 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321843 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321845 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321848 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321851 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321853 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321856 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.323574 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.321858 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322601 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322609 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322613 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322617 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322620 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322623 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322627 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322630 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322634 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322637 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322640 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322643 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322646 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322649 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322652 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322654 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322657 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322660 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.324154 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322663 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322665 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322668 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322671 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322674 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322677 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322680 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322682 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322685 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322687 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322691 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322694 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322696 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322699 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322702 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322705 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322707 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322710 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322712 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322715 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.324641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322718 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322721 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322723 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322725 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322728 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322731 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322733 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322735 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322738 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322741 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322744 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322747 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322750 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322752 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322756 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322758 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322761 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322764 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322767 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322770 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.325126 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322773 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322775 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322778 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322780 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322783 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322786 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322788 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322790 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322793 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322795 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322798 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322800 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322803 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322805 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322808 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322811 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322814 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322816 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322819 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322821 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.325639 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322824 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322827 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322829 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322832 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322834 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322837 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322840 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322842 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322845 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322848 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322850 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322853 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322855 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322858 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322861 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322864 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322866 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322869 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322872 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322875 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.326128 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.322877 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323278 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323283 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323287 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323290 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323293 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323296 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323314 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323317 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323320 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323324 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323327 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323330 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323333 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323336 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323339 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323341 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323344 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323347 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323350 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.326629 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323352 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323355 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323358 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323361 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323363 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323365 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323368 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323371 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323373 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323376 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323379 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323381 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323384 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323386 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323389 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323391 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323393 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323396 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323399 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.327143 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323402 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323404 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323407 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323410 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323412 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323415 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323417 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323420 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323423 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323425 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323430 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323434 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323437 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323440 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323442 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323445 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323448 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323451 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323453 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323456 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.327613 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323459 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323461 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323464 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323466 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323469 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323472 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323474 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323476 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323479 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323481 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323484 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323486 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323493 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323495 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323498 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323501 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323503 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323506 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323509 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323512 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.328118 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323514 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323517 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323521 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323524 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323527 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323530 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323533 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323536 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323538 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323540 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323544 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323547 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323550 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323553 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323555 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323558 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323561 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323563 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323566 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.328653 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323569 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.323571 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323643 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323650 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323656 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323661 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323666 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323669 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323674 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323679 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323682 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323686 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323689 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323692 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323696 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323699 2583 flags.go:64] FLAG: --cgroup-root="" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323702 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323705 2583 flags.go:64] FLAG: --client-ca-file="" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323708 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323711 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323716 2583 flags.go:64] FLAG: --cluster-domain="" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323719 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323722 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323725 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 22:04:15.329113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323729 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323732 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323735 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323738 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323742 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323745 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323748 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323751 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323757 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323760 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323764 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323767 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323771 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323774 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323777 2583 flags.go:64] FLAG: --enable-server="true" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323781 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323785 2583 flags.go:64] FLAG: --event-burst="100" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323788 2583 flags.go:64] FLAG: --event-qps="50" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323791 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323794 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323797 2583 flags.go:64] FLAG: --eviction-hard="" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323800 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323803 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323808 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323811 2583 flags.go:64] FLAG: --eviction-soft="" Jul 02 22:04:15.329688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323814 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323817 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323821 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323823 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323826 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323830 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323833 2583 flags.go:64] FLAG: --feature-gates="" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323837 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323840 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323843 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323846 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323849 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323852 2583 flags.go:64] FLAG: --help="false" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323855 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323858 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323862 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323865 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323869 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323872 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323875 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323878 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323881 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323884 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 22:04:15.330371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323887 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323890 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323893 2583 flags.go:64] FLAG: --kube-reserved="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323896 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323899 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323902 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323905 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323910 2583 flags.go:64] FLAG: --lock-file="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323913 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323916 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323919 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323925 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323928 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323931 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323933 2583 flags.go:64] FLAG: --logging-format="text" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323936 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323939 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323942 2583 flags.go:64] FLAG: --manifest-url="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323945 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323949 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323952 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323956 2583 flags.go:64] FLAG: --max-pods="110" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323959 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323962 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323966 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 22:04:15.330966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323969 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323972 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323975 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323978 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323985 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323988 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323991 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323994 2583 flags.go:64] FLAG: --pod-cidr="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.323998 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324004 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324007 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324010 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324013 2583 flags.go:64] FLAG: --port="10250" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324016 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324020 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab61323a1de40670" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324023 2583 flags.go:64] FLAG: --qos-reserved="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324026 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324029 2583 flags.go:64] FLAG: --register-node="true" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324040 2583 flags.go:64] FLAG: --register-with-taints="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324045 2583 flags.go:64] FLAG: --registry-burst="10" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324048 2583 flags.go:64] FLAG: --registry-qps="5" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324051 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324054 2583 flags.go:64] FLAG: --reserved-memory="" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324057 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324060 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 22:04:15.331585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324063 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324066 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324069 2583 flags.go:64] FLAG: --runonce="false" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324072 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324075 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324078 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324080 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324085 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324088 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324091 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324094 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324097 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324100 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324103 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324105 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324108 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324112 2583 flags.go:64] FLAG: --system-cgroups="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324115 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324120 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324123 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324126 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324132 2583 flags.go:64] FLAG: --tls-min-version="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324135 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324137 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324140 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 22:04:15.332212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324143 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324146 2583 flags.go:64] FLAG: --v="2" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324150 2583 flags.go:64] FLAG: --version="false" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324154 2583 flags.go:64] FLAG: --vmodule="" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324158 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324162 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324277 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324281 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324285 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324288 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324291 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324293 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324296 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324311 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324315 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324318 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324321 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324323 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324326 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324329 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324332 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.332843 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324334 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324337 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324340 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324343 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324346 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324348 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324351 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324356 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324359 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324362 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324365 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324367 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324371 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324373 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324376 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324379 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324381 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324384 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324387 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324391 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.333361 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324393 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324396 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324400 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324402 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324405 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324409 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324411 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324414 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324416 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324419 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324421 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324424 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324426 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324429 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324432 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324435 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324438 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324440 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324444 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.333892 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324449 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324452 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324455 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324458 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324461 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324464 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324467 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324470 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324472 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324475 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324478 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324480 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324483 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324486 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324488 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324492 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324495 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324499 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324503 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.334752 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324506 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324509 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324512 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324515 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324518 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324521 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324523 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324526 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324528 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324531 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324533 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324536 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324539 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324543 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324545 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324547 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324550 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324553 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324555 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324558 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.335378 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324560 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324563 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324565 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324568 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324570 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.324573 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.324579 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.332051 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.332069 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.332950 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.332958 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333017 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333023 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333026 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333029 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.335887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333032 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333035 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333038 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333041 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333044 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333046 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333049 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333052 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333055 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333058 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333060 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333063 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333066 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333069 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333072 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333074 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333077 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333080 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333082 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333085 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.336276 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333088 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333090 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333093 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333096 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333098 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333102 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333107 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333111 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333114 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333117 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333119 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333122 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333124 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333127 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333129 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333132 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333134 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333138 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333140 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.336791 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333142 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333145 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333147 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333150 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333153 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333155 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333158 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333160 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333163 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333165 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333168 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333171 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333173 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333176 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333178 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333181 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333183 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333187 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333190 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333201 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.337258 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333204 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333207 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333209 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333212 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333216 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333219 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333222 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333225 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333228 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333231 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333233 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333237 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333240 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333243 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333246 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333249 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333252 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333255 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333258 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.337771 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333261 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333263 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333266 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333268 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333271 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333273 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333276 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333278 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333281 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333283 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333286 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333290 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333292 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333295 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333308 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333311 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.338234 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333314 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.333320 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333437 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333443 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333446 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333448 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333451 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333454 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333457 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333459 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333462 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333465 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333467 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333470 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333472 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 22:04:15.338641 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333475 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333477 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333480 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333482 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333485 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333487 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333490 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333492 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333495 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333497 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333500 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333503 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333505 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333509 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333512 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333515 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333517 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333520 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333522 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 22:04:15.339016 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333526 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333530 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333533 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333536 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333539 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333542 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333544 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333547 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333550 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333553 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333555 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333558 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333561 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333563 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333566 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333568 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333571 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333573 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333576 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333578 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 22:04:15.339508 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333581 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333583 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333587 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333590 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333593 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333596 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333599 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333601 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333604 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333607 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333609 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333612 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333615 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333617 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333620 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333622 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333626 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333629 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333632 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 22:04:15.339997 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333635 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333638 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333640 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333643 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333645 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333648 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333650 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333653 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333655 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333658 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333660 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333663 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333666 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333668 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333671 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333673 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333676 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333678 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333681 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 22:04:15.340492 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333684 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333686 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333689 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333691 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333694 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333697 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333699 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333702 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:15.333704 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.333710 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 22:04:15.340951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.335709 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 22:04:15.376160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.376125 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 22:04:15.377755 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.377739 2583 server.go:1019] "Starting client certificate rotation" Jul 02 22:04:15.377866 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.377848 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 22:04:15.377919 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.377909 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 22:04:15.380195 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.380179 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 22:04:15.385647 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.385617 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 22:04:15.397285 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.397257 2583 log.go:25] "Validated CRI v1 runtime API" Jul 02 22:04:15.400168 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.400149 2583 log.go:25] "Validated CRI v1 image API" Jul 02 22:04:15.401188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.401172 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 22:04:15.406126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.406100 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 22:04:15.406649 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.406630 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aba74950-873a-462f-84fa-47f093c797e8:/dev/nvme0n1p3 bced387e-4048-4dc1-bbd4-8e9fcfb3aba5:/dev/nvme0n1p4] Jul 02 22:04:15.406709 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.406649 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 22:04:15.411875 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.411753 2583 manager.go:216] Machine: {Timestamp:2026-07-02 22:04:15.410313961 +0000 UTC m=+0.394526033 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099013 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24299a1639290e6b2cc1c23a364b06 SystemUUID:ec24299a-1639-290e-6b2c-c1c23a364b06 BootID:d9f2b842-fa76-4630-b55b-fd9aa51432c0 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:28:a8:54:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:28:a8:54:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:04:b7:28:b7:12 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 22:04:15.411875 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.411873 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 22:04:15.411993 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.411967 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 22:04:15.414920 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.414881 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 22:04:15.415074 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.414923 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 22:04:15.416293 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.416279 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 22:04:15.416350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.416295 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 22:04:15.416350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.416314 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 22:04:15.416876 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.416864 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 22:04:15.418160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.418147 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 22:04:15.421873 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.421858 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 02 22:04:15.422005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.421996 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 22:04:15.422039 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422034 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 02 22:04:15.422069 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422044 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 22:04:15.422069 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422060 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 22:04:15.422130 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422070 2583 kubelet.go:398] "Adding apiserver pod source" Jul 02 22:04:15.422130 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422081 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 22:04:15.422495 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422477 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 22:04:15.422495 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.422498 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 22:04:15.426866 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.426847 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 22:04:15.428310 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.428287 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 22:04:15.428373 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.428337 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 22:04:15.429854 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429841 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429860 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429867 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429873 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429883 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429893 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429905 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 22:04:15.429910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429911 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 22:04:15.430088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429918 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 22:04:15.430088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429924 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 22:04:15.430088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429933 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 22:04:15.430088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.429938 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 22:04:15.430905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.430894 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 22:04:15.430939 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.430906 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 22:04:15.434327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.434296 2583 server.go:1268] "Started kubelet" Jul 02 22:04:15.434441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.434392 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 22:04:15.434757 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.434428 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 22:04:15.434832 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.434757 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 22:04:15.434832 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.434798 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 22:04:15.434941 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.434890 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 22:04:15.435008 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.434993 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 22:04:15.435283 ip-10-0-128-202 systemd[1]: Started Kubernetes Kubelet. Jul 02 22:04:15.436349 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.436331 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 02 22:04:15.436633 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.436618 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 22:04:15.440445 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.440417 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 22:04:15.441279 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.440262 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-202.ec2.internal.18be9834e51347ac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-202.ec2.internal,UID:ip-10-0-128-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-202.ec2.internal,},FirstTimestamp:2026-07-02 22:04:15.434262444 +0000 UTC m=+0.418474516,LastTimestamp:2026-07-02 22:04:15.434262444 +0000 UTC m=+0.418474516,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-202.ec2.internal,}" Jul 02 22:04:15.442242 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.442228 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 22:04:15.442294 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.442246 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 22:04:15.442360 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.442341 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.442844 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.442828 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 22:04:15.442941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.442853 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 22:04:15.442941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.442867 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 22:04:15.443050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443009 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 02 22:04:15.443050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443021 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 02 22:04:15.443223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443202 2583 factory.go:55] Registering systemd factory Jul 02 22:04:15.443340 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443247 2583 factory.go:223] Registration of the systemd container factory successfully Jul 02 22:04:15.443465 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443453 2583 factory.go:153] Registering CRI-O factory Jul 02 22:04:15.443518 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443466 2583 factory.go:223] Registration of the crio container factory successfully Jul 02 22:04:15.443567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443518 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 22:04:15.443567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443542 2583 factory.go:103] Registering Raw factory Jul 02 22:04:15.443567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.443556 2583 manager.go:1196] Started watching for new ooms in manager Jul 02 22:04:15.444223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.444207 2583 manager.go:349] Starting recovery of all containers Jul 02 22:04:15.448554 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.448521 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 22:04:15.448554 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.448540 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 22:04:15.455502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.455266 2583 manager.go:354] Recovery completed Jul 02 22:04:15.456468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.456447 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk8kz" Jul 02 22:04:15.459623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.459611 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.461565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.461431 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.461666 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.461585 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.461666 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.461601 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.462227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.462211 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 22:04:15.462227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.462222 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 22:04:15.462430 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.462239 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 02 22:04:15.462430 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.462271 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.463063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.463042 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.463176 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.463073 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.463176 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.463087 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.463875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.463811 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-202.ec2.internal.18be9834e6b3e34b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-202.ec2.internal,UID:ip-10-0-128-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-202.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-202.ec2.internal,},FirstTimestamp:2026-07-02 22:04:15.461565259 +0000 UTC m=+0.445777336,LastTimestamp:2026-07-02 22:04:15.461565259 +0000 UTC m=+0.445777336,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-202.ec2.internal,}" Jul 02 22:04:15.464431 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.464410 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk8kz" Jul 02 22:04:15.464514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.464423 2583 policy_none.go:49] "None policy: Start" Jul 02 22:04:15.464514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.464464 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 22:04:15.464514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.464483 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 22:04:15.466451 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.466440 2583 policy_none.go:47] "Start" Jul 02 22:04:15.504954 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.504928 2583 manager.go:337] "Starting Device Plugin manager" Jul 02 22:04:15.505119 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.504974 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 22:04:15.505119 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.504989 2583 server.go:87] "Starting device plugin registration server" Jul 02 22:04:15.505703 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.505291 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 22:04:15.505703 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.505336 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 22:04:15.505703 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.505454 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 22:04:15.505703 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.505523 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 22:04:15.505703 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.505531 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 22:04:15.507900 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.507868 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 22:04:15.508012 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.507928 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.593428 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.593333 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 22:04:15.594634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.594604 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 22:04:15.594691 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.594662 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 22:04:15.594729 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.594690 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 22:04:15.594771 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.594734 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 22:04:15.596761 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.596738 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:15.605826 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.605807 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.606709 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.606689 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.606771 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.606727 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.606771 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.606738 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.606771 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.606761 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.617743 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.617716 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.617743 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.617744 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-202.ec2.internal\": node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.638805 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.638773 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.695023 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.694966 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal"] Jul 02 22:04:15.695107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.695080 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.697604 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.697583 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.697735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.697622 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.697735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.697638 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.700099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.700084 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.700224 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.700208 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.700270 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.700244 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.701015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.700997 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.701102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.701018 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.701102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.701031 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.701102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.701037 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.701102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.701046 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.701102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.701048 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.703367 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.703351 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.703367 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.703378 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 22:04:15.708966 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.708948 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientMemory" Jul 02 22:04:15.709026 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.708977 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 22:04:15.709026 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.709000 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeHasSufficientPID" Jul 02 22:04:15.723108 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.723080 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-202.ec2.internal\" not found" node="ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.729613 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.729594 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-202.ec2.internal\" not found" node="ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.739681 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.739657 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.840577 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.840521 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.845318 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.845250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.845318 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.845284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.845464 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.845309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/948f3bc00963a2485050a8f8925dd13e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-202.ec2.internal\" (UID: \"948f3bc00963a2485050a8f8925dd13e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.941084 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:15.941026 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:15.945733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/948f3bc00963a2485050a8f8925dd13e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-202.ec2.internal\" (UID: \"948f3bc00963a2485050a8f8925dd13e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.945806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945742 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.945806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945762 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.945806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945803 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.945905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945806 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31533b5d6172412ca612eb1395690822-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal\" (UID: \"31533b5d6172412ca612eb1395690822\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:15.945905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:15.945840 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/948f3bc00963a2485050a8f8925dd13e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-202.ec2.internal\" (UID: \"948f3bc00963a2485050a8f8925dd13e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:16.027154 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.027108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:16.033545 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.033510 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:16.041240 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.041217 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.142561 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.142459 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.244575 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.244524 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.253693 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.253664 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:16.345231 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.345181 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.376840 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.376805 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 22:04:16.376982 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.376964 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 22:04:16.377044 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.376988 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 22:04:16.445781 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.445708 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.467706 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.467656 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 21:59:16 +0000 UTC" deadline="2028-04-11 20:23:00.054502082 +0000 UTC" Jul 02 22:04:16.467706 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.467698 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15574h18m43.586809546s" Jul 02 22:04:16.545827 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.545797 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.587295 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.587254 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:16.645985 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.645944 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.670417 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:16.670383 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948f3bc00963a2485050a8f8925dd13e.slice/crio-0536edc100d2b35a5236b2172892a483e1cf0ef8fcdb1024e5034a5818875661 WatchSource:0}: Error finding container 0536edc100d2b35a5236b2172892a483e1cf0ef8fcdb1024e5034a5818875661: Status 404 returned error can't find the container with id 0536edc100d2b35a5236b2172892a483e1cf0ef8fcdb1024e5034a5818875661 Jul 02 22:04:16.672959 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:16.672938 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31533b5d6172412ca612eb1395690822.slice/crio-10f938dd3ffcf8ed1660d0b27b762ae328e9fef4d383986b1564e8181943759e WatchSource:0}: Error finding container 10f938dd3ffcf8ed1660d0b27b762ae328e9fef4d383986b1564e8181943759e: Status 404 returned error can't find the container with id 10f938dd3ffcf8ed1660d0b27b762ae328e9fef4d383986b1564e8181943759e Jul 02 22:04:16.677075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:16.677059 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:04:16.746207 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.746109 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.849296 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.849254 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:16.950167 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:16.950120 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-202.ec2.internal\" not found" Jul 02 22:04:17.021336 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.021245 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:17.042872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.042831 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" Jul 02 22:04:17.058206 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.058177 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 22:04:17.059505 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.059488 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" Jul 02 22:04:17.068985 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.068965 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 22:04:17.424650 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.424565 2583 apiserver.go:52] "Watching apiserver" Jul 02 22:04:17.433458 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.433418 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:17.434792 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.434766 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 22:04:17.435572 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.435261 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-7sz94","kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal","openshift-multus/multus-additional-cni-plugins-llrgq","openshift-multus/multus-h2qzt","openshift-network-operator/iptables-alerter-p5bgk","openshift-ovn-kubernetes/ovnkube-node-chpms","kube-system/global-pull-secret-syncer-q55pk","openshift-cluster-node-tuning-operator/tuned-5q49n","openshift-dns/node-resolver-zmxlq","openshift-image-registry/node-ca-fqvf7","openshift-multus/network-metrics-daemon-sqlzp","openshift-network-diagnostics/network-check-target-n7k68"] Jul 02 22:04:17.438301 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.438019 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.440653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.440493 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.440653 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.440589 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:17.441521 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.441496 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.441645 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.441589 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.441709 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.441505 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnc8h\"" Jul 02 22:04:17.443253 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.442982 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 22:04:17.443370 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.443267 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.445523 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.445478 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:17.445660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.445553 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:17.446459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.446064 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 22:04:17.446459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.446099 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.446459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.446133 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7r9lk\"" Jul 02 22:04:17.447872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.447568 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.447973 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.447930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.450406 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.450252 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.452285 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.452265 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 22:04:17.452434 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.452417 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 22:04:17.453002 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.452980 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.453241 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453218 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 22:04:17.453351 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453258 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.453351 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453051 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.453484 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453459 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453545 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453490 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453545 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453521 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:17.453659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453548 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-kubernetes\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453575 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-var-lib-kubelet\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-device-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453681 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tlt\" (UniqueName: \"kubernetes.io/projected/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kube-api-access-z2tlt\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453723 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysconfig\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-sys\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453788 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-lib-modules\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-tmp\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453838 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgsg7\" (UniqueName: \"kubernetes.io/projected/4a4ca393-cf0e-4866-b73a-76ad7790a42d-kube-api-access-hgsg7\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453911 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-conf\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.453932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453935 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-systemd\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453940 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n9kxl\"" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-run\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-tuned\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453990 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.454007 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w52cn\" (UniqueName: \"kubernetes.io/projected/a32e4e26-91ab-40d4-8c13-db3a91c646e6-kube-api-access-w52cn\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.454040 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.454066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-modprobe-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.454091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.454113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-host\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.453938 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w4ppc\"" Jul 02 22:04:17.454445 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.453210 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:17.455700 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.455682 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.456654 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.456626 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 22:04:17.458401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.458301 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 22:04:17.458586 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.458565 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hjj4f\"" Jul 02 22:04:17.458667 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.458586 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.458815 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.458799 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.458995 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.458974 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.461391 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.461365 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.461651 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.461632 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 22:04:17.461740 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.461725 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 22:04:17.462000 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.461975 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-766dq\"" Jul 02 22:04:17.462087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.462013 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.462266 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.462246 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 22:04:17.463290 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.463270 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 22:04:17.463395 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.463301 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.464469 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.464445 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 22:04:17.464554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.464482 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 22:04:17.464624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.464605 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8svg2\"" Jul 02 22:04:17.470100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.469498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.472956 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.472752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cp4sh\"" Jul 02 22:04:17.473083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.473064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.477051 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.476273 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.477051 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.476587 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.479766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.479506 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 22:04:17.479766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.479538 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-krrc6\"" Jul 02 22:04:17.480136 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.479782 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 22:04:17.480136 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.479790 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmg7l" Jul 02 22:04:17.480136 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.479854 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 22:04:17.487638 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.487609 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmg7l" Jul 02 22:04:17.544913 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.544883 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 22:04:17.555081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-host-slash\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.555081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555082 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-etc-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.555311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-bin\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.555311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-os-release\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555236 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-multus-certs\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-log-socket\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.555311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-netd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555350 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555455 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-hostroot\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555482 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555491 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-kubernetes\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-var-lib-kubelet\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555557 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-kubernetes\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.555609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555581 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555614 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-var-lib-kubelet\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-binary-copy\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555708 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555717 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-system-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555762 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9r6\" (UniqueName: \"kubernetes.io/projected/fb7ce651-2b47-47dd-8464-bce148a8b057-kube-api-access-pd9r6\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-k8s-cni-cncf-io\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgsg7\" (UniqueName: \"kubernetes.io/projected/4a4ca393-cf0e-4866-b73a-76ad7790a42d-kube-api-access-hgsg7\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69304590-8e37-4dc2-a16a-b8e8ea9208a5-serviceca\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555953 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-system-cni-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.555992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.555985 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-socket-dir-parent\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-conf\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-run\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556129 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69304590-8e37-4dc2-a16a-b8e8ea9208a5-host\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-os-release\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556179 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-cnibin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556206 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z48bw\" (UniqueName: \"kubernetes.io/projected/f8c728c9-da49-4fba-9184-e4c93b0b197e-kube-api-access-z48bw\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-cnibin\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.556370 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556446 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-run\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.556502 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556480 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-multus\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556508 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-conf-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-slash\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-netns\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-script-lib\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysconfig\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-systemd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556693 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-config\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556697 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-conf\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556721 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg92s\" (UniqueName: \"kubernetes.io/projected/c25580bb-6095-4326-91af-eef364ec7f4d-kube-api-access-tg92s\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.556789 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:18.056763369 +0000 UTC m=+3.040975455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556806 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysconfig\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-tuned\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w52cn\" (UniqueName: \"kubernetes.io/projected/a32e4e26-91ab-40d4-8c13-db3a91c646e6-kube-api-access-w52cn\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556940 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-etc-kubernetes\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.557181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.556968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-var-lib-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02517e0f-1525-477c-bee1-3bc04b0ceebb-tmp-dir\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557071 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-host\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-host\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-dbus\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be2968ab-3e35-4ac0-9403-378a0f2fe535-konnectivity-ca\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557249 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-systemd-units\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557297 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-sysctl-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-ovn\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-device-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557384 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-kubelet-config\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557453 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-netns\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557462 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-device-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.557988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557489 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-sys\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557517 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-lib-modules\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557571 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-sys\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557579 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbrc\" (UniqueName: \"kubernetes.io/projected/69304590-8e37-4dc2-a16a-b8e8ea9208a5-kube-api-access-5jbrc\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557611 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt8mk\" (UniqueName: \"kubernetes.io/projected/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-kube-api-access-zt8mk\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-node-log\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557627 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557659 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-lib-modules\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-env-overrides\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557774 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-systemd\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557854 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-systemd\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.557928 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25580bb-6095-4326-91af-eef364ec7f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558004 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02517e0f-1525-477c-bee1-3bc04b0ceebb-hosts-file\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4n6\" (UniqueName: \"kubernetes.io/projected/02517e0f-1525-477c-bee1-3bc04b0ceebb-kube-api-access-dv4n6\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.558838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558061 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/555df824-6fc1-4b40-a8dc-c6f5568d19c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-kubelet\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-modprobe-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-cni-binary-copy\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558172 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-iptables-alerter-script\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-kubelet\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tlt\" (UniqueName: \"kubernetes.io/projected/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kube-api-access-z2tlt\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558285 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558288 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-modprobe-d\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-daemon-config\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-tmp\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558518 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be2968ab-3e35-4ac0-9403-378a0f2fe535-agent-certs\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558588 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.559690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.558615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-bin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.561724 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.561666 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-tmp\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.561896 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.561732 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a32e4e26-91ab-40d4-8c13-db3a91c646e6-etc-tuned\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.569306 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.568966 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:17.569306 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.569004 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:17.569306 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.569021 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:17.569306 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.569093 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:18.069070717 +0000 UTC m=+3.053282780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:17.571216 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.571183 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgsg7\" (UniqueName: \"kubernetes.io/projected/4a4ca393-cf0e-4866-b73a-76ad7790a42d-kube-api-access-hgsg7\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:17.572047 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.571996 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tlt\" (UniqueName: \"kubernetes.io/projected/555df824-6fc1-4b40-a8dc-c6f5568d19c9-kube-api-access-z2tlt\") pod \"aws-ebs-csi-driver-node-7kbfh\" (UID: \"555df824-6fc1-4b40-a8dc-c6f5568d19c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.573244 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.573196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w52cn\" (UniqueName: \"kubernetes.io/projected/a32e4e26-91ab-40d4-8c13-db3a91c646e6-kube-api-access-w52cn\") pod \"tuned-5q49n\" (UID: \"a32e4e26-91ab-40d4-8c13-db3a91c646e6\") " pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.601254 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.601188 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" event={"ID":"31533b5d6172412ca612eb1395690822","Type":"ContainerStarted","Data":"10f938dd3ffcf8ed1660d0b27b762ae328e9fef4d383986b1564e8181943759e"} Jul 02 22:04:17.603121 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.603079 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" event={"ID":"948f3bc00963a2485050a8f8925dd13e","Type":"ContainerStarted","Data":"0536edc100d2b35a5236b2172892a483e1cf0ef8fcdb1024e5034a5818875661"} Jul 02 22:04:17.659416 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659361 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-cnibin\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.659416 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659408 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-multus\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.659416 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-conf-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659499 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-cnibin\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659499 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-conf-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659529 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-multus\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659565 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-slash\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-netns\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659648 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-slash\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659650 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-script-lib\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-netns\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-systemd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.659738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659725 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-config\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659749 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg92s\" (UniqueName: \"kubernetes.io/projected/c25580bb-6095-4326-91af-eef364ec7f4d-kube-api-access-tg92s\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-systemd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-etc-kubernetes\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659819 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-etc-kubernetes\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-var-lib-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659930 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.659958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02517e0f-1525-477c-bee1-3bc04b0ceebb-tmp-dir\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-dbus\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be2968ab-3e35-4ac0-9403-378a0f2fe535-konnectivity-ca\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-systemd-units\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-ovn\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-kubelet-config\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.660281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660262 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-netns\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.661096 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbrc\" (UniqueName: \"kubernetes.io/projected/69304590-8e37-4dc2-a16a-b8e8ea9208a5-kube-api-access-5jbrc\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.661096 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660329 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8mk\" (UniqueName: \"kubernetes.io/projected/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-kube-api-access-zt8mk\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.661096 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660412 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-systemd-units\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.661096 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660430 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-node-log\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.661508 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.661481 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-script-lib\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.661891 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.660468 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-var-lib-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.662097 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-ovnkube-config\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.662195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-env-overrides\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.662250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.662287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25580bb-6095-4326-91af-eef364ec7f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.662342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02517e0f-1525-477c-bee1-3bc04b0ceebb-hosts-file\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4n6\" (UniqueName: \"kubernetes.io/projected/02517e0f-1525-477c-bee1-3bc04b0ceebb-kube-api-access-dv4n6\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-kubelet\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-cni-binary-copy\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-iptables-alerter-script\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663742 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-kubelet\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663761 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-kubelet\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663820 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663826 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-kubelet\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663849 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02517e0f-1525-477c-bee1-3bc04b0ceebb-tmp-dir\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-daemon-config\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.664656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663955 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be2968ab-3e35-4ac0-9403-378a0f2fe535-agent-certs\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-kubelet-config\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.663995 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-bin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-host-slash\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664099 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-etc-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-bin\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-os-release\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-multus-certs\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664230 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-log-socket\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664256 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02517e0f-1525-477c-bee1-3bc04b0ceebb-hosts-file\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-netd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664310 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664318 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-netd\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664346 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-hostroot\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664430 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664554 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be2968ab-3e35-4ac0-9403-378a0f2fe535-konnectivity-ca\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664590 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.665695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664752 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-daemon-config\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664792 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c25580bb-6095-4326-91af-eef364ec7f4d-env-overrides\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664906 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-etc-openvswitch\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.664913 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.664960 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-var-lib-cni-bin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:17.664996 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:18.164969387 +0000 UTC m=+3.149181464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-binary-copy\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665050 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-cni-bin\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-system-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665170 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-iptables-alerter-script\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665287 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-netns\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665308 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-host-slash\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665072 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-system-cni-dir\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665517 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665638 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-os-release\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665693 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-node-log\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665797 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.666679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.665839 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c995aee-a8c5-4d10-b49e-47da30a93d9b-dbus\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.666168 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.666306 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-multus-certs\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.666419 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7ce651-2b47-47dd-8464-bce148a8b057-cni-binary-copy\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.666351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9r6\" (UniqueName: \"kubernetes.io/projected/fb7ce651-2b47-47dd-8464-bce148a8b057-kube-api-access-pd9r6\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.666930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667055 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-run-ovn\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-k8s-cni-cncf-io\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667081 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c728c9-da49-4fba-9184-e4c93b0b197e-cni-binary-copy\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667143 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-host-run-k8s-cni-cncf-io\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69304590-8e37-4dc2-a16a-b8e8ea9208a5-serviceca\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667247 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-hostroot\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667240 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c25580bb-6095-4326-91af-eef364ec7f4d-log-socket\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.667570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667532 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-system-cni-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667666 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-socket-dir-parent\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667718 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69304590-8e37-4dc2-a16a-b8e8ea9208a5-serviceca\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667737 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-system-cni-dir\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-multus-socket-dir-parent\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69304590-8e37-4dc2-a16a-b8e8ea9208a5-host\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-os-release\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667891 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69304590-8e37-4dc2-a16a-b8e8ea9208a5-host\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.667994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-cnibin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.668076 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c728c9-da49-4fba-9184-e4c93b0b197e-os-release\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.668079 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7ce651-2b47-47dd-8464-bce148a8b057-cnibin\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.668344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.668080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z48bw\" (UniqueName: \"kubernetes.io/projected/f8c728c9-da49-4fba-9184-e4c93b0b197e-kube-api-access-z48bw\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.670298 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.670268 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be2968ab-3e35-4ac0-9403-378a0f2fe535-agent-certs\") pod \"konnectivity-agent-7sz94\" (UID: \"be2968ab-3e35-4ac0-9403-378a0f2fe535\") " pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.672087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.672058 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25580bb-6095-4326-91af-eef364ec7f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.673185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.672997 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbrc\" (UniqueName: \"kubernetes.io/projected/69304590-8e37-4dc2-a16a-b8e8ea9208a5-kube-api-access-5jbrc\") pod \"node-ca-fqvf7\" (UID: \"69304590-8e37-4dc2-a16a-b8e8ea9208a5\") " pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:17.673185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.673138 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg92s\" (UniqueName: \"kubernetes.io/projected/c25580bb-6095-4326-91af-eef364ec7f4d-kube-api-access-tg92s\") pod \"ovnkube-node-chpms\" (UID: \"c25580bb-6095-4326-91af-eef364ec7f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.674796 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.674726 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4n6\" (UniqueName: \"kubernetes.io/projected/02517e0f-1525-477c-bee1-3bc04b0ceebb-kube-api-access-dv4n6\") pod \"node-resolver-zmxlq\" (UID: \"02517e0f-1525-477c-bee1-3bc04b0ceebb\") " pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.675093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.675067 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt8mk\" (UniqueName: \"kubernetes.io/projected/99d8e01c-490a-47c8-90cc-3afa75bbf4ba-kube-api-access-zt8mk\") pod \"iptables-alerter-p5bgk\" (UID: \"99d8e01c-490a-47c8-90cc-3afa75bbf4ba\") " pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.675785 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.675767 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9r6\" (UniqueName: \"kubernetes.io/projected/fb7ce651-2b47-47dd-8464-bce148a8b057-kube-api-access-pd9r6\") pod \"multus-h2qzt\" (UID: \"fb7ce651-2b47-47dd-8464-bce148a8b057\") " pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.681127 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.681099 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z48bw\" (UniqueName: \"kubernetes.io/projected/f8c728c9-da49-4fba-9184-e4c93b0b197e-kube-api-access-z48bw\") pod \"multus-additional-cni-plugins-llrgq\" (UID: \"f8c728c9-da49-4fba-9184-e4c93b0b197e\") " pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.754160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.754119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5q49n" Jul 02 22:04:17.774544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.774509 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" Jul 02 22:04:17.785749 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.785722 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-llrgq" Jul 02 22:04:17.789582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.789561 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2qzt" Jul 02 22:04:17.798475 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.798444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p5bgk" Jul 02 22:04:17.806683 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.806633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:17.817531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.817507 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:17.824503 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.824477 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zmxlq" Jul 02 22:04:17.831468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:17.831439 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fqvf7" Jul 02 22:04:18.071245 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.071148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:18.071245 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.071188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071341 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071361 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071386 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071399 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071414 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:19.071393667 +0000 UTC m=+4.055605736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:18.071510 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.071439 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:19.071427481 +0000 UTC m=+4.055639548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:18.172213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.172172 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:18.172439 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.172313 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:18.172439 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.172386 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:19.172370248 +0000 UTC m=+4.156582320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:18.492239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.492147 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:59:17 +0000 UTC" deadline="2028-04-11 12:51:23.952299072 +0000 UTC" Jul 02 22:04:18.492239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.492181 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15566h47m5.460121014s" Jul 02 22:04:18.595695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.595645 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:18.595899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.595699 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:18.595899 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.595791 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:18.596029 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:18.595949 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:18.788522 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:18.788423 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 22:04:18.870658 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.870623 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c728c9_da49_4fba_9184_e4c93b0b197e.slice/crio-0967b137243a23f935c7c1bd826b09632dcff9bc724bd297d124cb52ee1b3f9a WatchSource:0}: Error finding container 0967b137243a23f935c7c1bd826b09632dcff9bc724bd297d124cb52ee1b3f9a: Status 404 returned error can't find the container with id 0967b137243a23f935c7c1bd826b09632dcff9bc724bd297d124cb52ee1b3f9a Jul 02 22:04:18.871832 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.871802 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02517e0f_1525_477c_bee1_3bc04b0ceebb.slice/crio-f5c66eb800cc58f217741b7f1c23afad86aa7a49340c7d5693c1184ba0015e37 WatchSource:0}: Error finding container f5c66eb800cc58f217741b7f1c23afad86aa7a49340c7d5693c1184ba0015e37: Status 404 returned error can't find the container with id f5c66eb800cc58f217741b7f1c23afad86aa7a49340c7d5693c1184ba0015e37 Jul 02 22:04:18.873138 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.873043 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555df824_6fc1_4b40_a8dc_c6f5568d19c9.slice/crio-bf265d6f01f5325ec37d6a30392219a4478a477a62c1b55333960c38140e62e2 WatchSource:0}: Error finding container bf265d6f01f5325ec37d6a30392219a4478a477a62c1b55333960c38140e62e2: Status 404 returned error can't find the container with id bf265d6f01f5325ec37d6a30392219a4478a477a62c1b55333960c38140e62e2 Jul 02 22:04:18.877008 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.876686 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d8e01c_490a_47c8_90cc_3afa75bbf4ba.slice/crio-4f377ac163b2837d4c17d02590878b69188b054df27a916dd080b03373b4335c WatchSource:0}: Error finding container 4f377ac163b2837d4c17d02590878b69188b054df27a916dd080b03373b4335c: Status 404 returned error can't find the container with id 4f377ac163b2837d4c17d02590878b69188b054df27a916dd080b03373b4335c Jul 02 22:04:18.878180 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.878084 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2968ab_3e35_4ac0_9403_378a0f2fe535.slice/crio-b89567eaccc8aeeeb3173736799cbc66bd5b1c091e85cba5dc277c430fcdcf20 WatchSource:0}: Error finding container b89567eaccc8aeeeb3173736799cbc66bd5b1c091e85cba5dc277c430fcdcf20: Status 404 returned error can't find the container with id b89567eaccc8aeeeb3173736799cbc66bd5b1c091e85cba5dc277c430fcdcf20 Jul 02 22:04:18.879077 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.878949 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69304590_8e37_4dc2_a16a_b8e8ea9208a5.slice/crio-252851426c4c87d127f99ab765ae8eb8fae93b208fdaeeb9176a80dc5dbb0c28 WatchSource:0}: Error finding container 252851426c4c87d127f99ab765ae8eb8fae93b208fdaeeb9176a80dc5dbb0c28: Status 404 returned error can't find the container with id 252851426c4c87d127f99ab765ae8eb8fae93b208fdaeeb9176a80dc5dbb0c28 Jul 02 22:04:18.880032 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.879846 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7ce651_2b47_47dd_8464_bce148a8b057.slice/crio-98266c3cd30263b6a9b3e3d8048a1c7091770d1df8a5a629f91bb6ad64410860 WatchSource:0}: Error finding container 98266c3cd30263b6a9b3e3d8048a1c7091770d1df8a5a629f91bb6ad64410860: Status 404 returned error can't find the container with id 98266c3cd30263b6a9b3e3d8048a1c7091770d1df8a5a629f91bb6ad64410860 Jul 02 22:04:18.880980 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.880758 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25580bb_6095_4326_91af_eef364ec7f4d.slice/crio-159a803bf5fd1b9b02fcdb1ef058232d4b2985b2a3b2ba596697fa45b719c188 WatchSource:0}: Error finding container 159a803bf5fd1b9b02fcdb1ef058232d4b2985b2a3b2ba596697fa45b719c188: Status 404 returned error can't find the container with id 159a803bf5fd1b9b02fcdb1ef058232d4b2985b2a3b2ba596697fa45b719c188 Jul 02 22:04:18.883051 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:18.882948 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32e4e26_91ab_40d4_8c13_db3a91c646e6.slice/crio-29ff3d82bb56a8bf60d29641bebf268de1e3e89a389025e01a0903d831abd2a2 WatchSource:0}: Error finding container 29ff3d82bb56a8bf60d29641bebf268de1e3e89a389025e01a0903d831abd2a2: Status 404 returned error can't find the container with id 29ff3d82bb56a8bf60d29641bebf268de1e3e89a389025e01a0903d831abd2a2 Jul 02 22:04:19.078749 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.078456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:19.078749 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.078698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:19.078749 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078617 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:19.078968 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078824 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:21.078803429 +0000 UTC m=+6.063015504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:19.078968 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078906 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:19.078968 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078923 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:19.078968 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078937 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:19.079144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.078997 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:21.078981951 +0000 UTC m=+6.063194016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:19.179103 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.179070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:19.179285 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.179259 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:19.179397 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.179383 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:21.179358467 +0000 UTC m=+6.163570545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:19.494171 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.493214 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:59:17 +0000 UTC" deadline="2028-02-25 08:33:15.796277973 +0000 UTC" Jul 02 22:04:19.494171 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.493259 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14458h28m56.303025055s" Jul 02 22:04:19.598868 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.598252 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:19.598868 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:19.598446 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:19.611689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.611593 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p5bgk" event={"ID":"99d8e01c-490a-47c8-90cc-3afa75bbf4ba","Type":"ContainerStarted","Data":"4f377ac163b2837d4c17d02590878b69188b054df27a916dd080b03373b4335c"} Jul 02 22:04:19.614102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.614028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zmxlq" event={"ID":"02517e0f-1525-477c-bee1-3bc04b0ceebb","Type":"ContainerStarted","Data":"f5c66eb800cc58f217741b7f1c23afad86aa7a49340c7d5693c1184ba0015e37"} Jul 02 22:04:19.619864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.618884 2583 generic.go:356] "Generic (PLEG): container finished" podID="31533b5d6172412ca612eb1395690822" containerID="9e010ea3da5dcfef1618cc3570d6493aaa56fa3d2cd24b806ae583a703d678c2" exitCode=0 Jul 02 22:04:19.619864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.618979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" event={"ID":"31533b5d6172412ca612eb1395690822","Type":"ContainerDied","Data":"9e010ea3da5dcfef1618cc3570d6493aaa56fa3d2cd24b806ae583a703d678c2"} Jul 02 22:04:19.633614 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.633478 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" event={"ID":"948f3bc00963a2485050a8f8925dd13e","Type":"ContainerStarted","Data":"7a79896d033bef8a20d73eb74a69f803a160f13337439b29d87886749e77ceb4"} Jul 02 22:04:19.640249 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.640213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7sz94" event={"ID":"be2968ab-3e35-4ac0-9403-378a0f2fe535","Type":"ContainerStarted","Data":"b89567eaccc8aeeeb3173736799cbc66bd5b1c091e85cba5dc277c430fcdcf20"} Jul 02 22:04:19.650127 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.650015 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fqvf7" event={"ID":"69304590-8e37-4dc2-a16a-b8e8ea9208a5","Type":"ContainerStarted","Data":"252851426c4c87d127f99ab765ae8eb8fae93b208fdaeeb9176a80dc5dbb0c28"} Jul 02 22:04:19.655561 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.655460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5q49n" event={"ID":"a32e4e26-91ab-40d4-8c13-db3a91c646e6","Type":"ContainerStarted","Data":"29ff3d82bb56a8bf60d29641bebf268de1e3e89a389025e01a0903d831abd2a2"} Jul 02 22:04:19.666665 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.666627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" event={"ID":"555df824-6fc1-4b40-a8dc-c6f5568d19c9","Type":"ContainerStarted","Data":"bf265d6f01f5325ec37d6a30392219a4478a477a62c1b55333960c38140e62e2"} Jul 02 22:04:19.682675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.681563 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerStarted","Data":"0967b137243a23f935c7c1bd826b09632dcff9bc724bd297d124cb52ee1b3f9a"} Jul 02 22:04:19.684902 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.684773 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"159a803bf5fd1b9b02fcdb1ef058232d4b2985b2a3b2ba596697fa45b719c188"} Jul 02 22:04:19.689095 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:19.689019 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2qzt" event={"ID":"fb7ce651-2b47-47dd-8464-bce148a8b057","Type":"ContainerStarted","Data":"98266c3cd30263b6a9b3e3d8048a1c7091770d1df8a5a629f91bb6ad64410860"} Jul 02 22:04:20.595708 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:20.594987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:20.595708 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:20.595126 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:20.595708 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:20.595557 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:20.595708 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:20.595653 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:20.706089 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:20.706052 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" event={"ID":"31533b5d6172412ca612eb1395690822","Type":"ContainerStarted","Data":"526d71591cec7a57bacd01eb9595fc8e4af851e5dfd6b08fd227dca72059df45"} Jul 02 22:04:20.725608 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:20.725539 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-202.ec2.internal" podStartSLOduration=3.725512562 podStartE2EDuration="3.725512562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:19.665276728 +0000 UTC m=+4.649488813" watchObservedRunningTime="2026-07-02 22:04:20.725512562 +0000 UTC m=+5.709724648" Jul 02 22:04:20.725846 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:20.725810 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-202.ec2.internal" podStartSLOduration=3.7257990899999998 podStartE2EDuration="3.72579909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:20.725410984 +0000 UTC m=+5.709623084" watchObservedRunningTime="2026-07-02 22:04:20.72579909 +0000 UTC m=+5.710011177" Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:21.096639 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:21.096694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.096856 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.096876 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.096891 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.096953 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:25.096932019 +0000 UTC m=+10.081144082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.097371 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:21.097492 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.097444 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:25.097425941 +0000 UTC m=+10.081638002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:21.199076 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:21.198353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:21.199076 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.198595 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:21.199076 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.198667 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:25.1986481 +0000 UTC m=+10.182860175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:21.596653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:21.596552 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:21.597145 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:21.596723 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:22.595682 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:22.595504 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:22.595682 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:22.595510 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:22.595682 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:22.595635 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:22.595991 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:22.595795 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:23.596330 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:23.595729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:23.596330 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:23.595878 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:24.595197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:24.595150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:24.595399 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:24.595284 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:24.595536 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:24.595456 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:24.595536 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:24.595525 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:25.135701 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:25.135661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:25.135719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.135845 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.135921 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.135926 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:33.13590297 +0000 UTC m=+18.120115031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.135939 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.135953 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:25.136144 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.136003 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:33.13598667 +0000 UTC m=+18.120198731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:25.236615 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:25.236573 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:25.236798 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.236712 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:25.236798 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.236776 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:33.236759634 +0000 UTC m=+18.220971696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:25.596222 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:25.596122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:25.596422 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:25.596262 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:26.595370 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:26.595333 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:26.596047 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:26.595339 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:26.596047 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:26.595481 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:26.596047 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:26.595574 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:27.596514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:27.595996 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:27.596514 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:27.596149 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:28.594976 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:28.594939 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:28.595155 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:28.594946 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:28.595155 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:28.595080 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:28.595155 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:28.595112 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:29.595178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:29.595143 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:29.595616 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:29.595314 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:30.595060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:30.595025 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:30.595231 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:30.595071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:30.595231 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:30.595156 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:30.595689 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:30.595284 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:30.733880 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:30.733844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7sz94" event={"ID":"be2968ab-3e35-4ac0-9403-378a0f2fe535","Type":"ContainerStarted","Data":"605faeb499744c0a41c5095a28e98c74f352b038064f37a7a40857268937d93f"} Jul 02 22:04:30.751150 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:30.751095 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7sz94" podStartSLOduration=3.95747706 podStartE2EDuration="14.751079478s" totalImagesPullingTime="10.793602418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:30.750497551 +0000 UTC m=+15.734709632" watchObservedRunningTime="2026-07-02 22:04:30.751079478 +0000 UTC m=+15.735291601" Jul 02 22:04:31.595925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:31.595889 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:31.596376 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:31.596057 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:31.738097 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:31.738001 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p5bgk" event={"ID":"99d8e01c-490a-47c8-90cc-3afa75bbf4ba","Type":"ContainerStarted","Data":"a6fdff34459b48dd9ae46e6e7c384327bd13bf2d4acdcbc88f38ef13ee630918"} Jul 02 22:04:31.754976 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:31.754914 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p5bgk" podStartSLOduration=4.960438807 podStartE2EDuration="15.754893817s" totalImagesPullingTime="10.79445501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:31.754525881 +0000 UTC m=+16.738737963" watchObservedRunningTime="2026-07-02 22:04:31.754893817 +0000 UTC m=+16.739105900" Jul 02 22:04:32.453669 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.453636 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:32.454283 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.454263 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:32.595059 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.595031 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:32.595258 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.595031 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:32.595258 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:32.595137 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:32.595258 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:32.595230 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:32.740508 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.740432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:32.741028 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:32.741012 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7sz94" Jul 02 22:04:33.197941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:33.197903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:33.197941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:33.197956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198090 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198119 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198140 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198153 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198160 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:04:49.198139351 +0000 UTC m=+34.182351425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:33.198262 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.198200 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:04:49.198184807 +0000 UTC m=+34.182396891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:33.299021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:33.298981 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:33.299190 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.299127 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:33.299250 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.299192 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:49.299175333 +0000 UTC m=+34.283387398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:33.595804 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:33.595711 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:33.595973 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:33.595844 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:34.595386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:34.595166 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:34.595823 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:34.595171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:34.595823 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:34.595474 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:34.595823 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:34.595567 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:35.596183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:35.596136 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:35.596668 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:35.596272 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:36.595058 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:36.595020 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:36.595235 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:36.595027 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:36.595235 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:36.595133 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:36.595355 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:36.595229 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:37.595788 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:37.595749 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:37.596232 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:37.595899 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:38.595453 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.595232 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:38.595685 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.595255 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:38.595685 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:38.595551 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:38.595685 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:38.595592 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:38.753734 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.753697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fqvf7" event={"ID":"69304590-8e37-4dc2-a16a-b8e8ea9208a5","Type":"ContainerStarted","Data":"7a5e33451004e9c41cba05d5e011a4d011912463e291748de6c6e630cd5ede0d"} Jul 02 22:04:38.755156 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.755131 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5q49n" event={"ID":"a32e4e26-91ab-40d4-8c13-db3a91c646e6","Type":"ContainerStarted","Data":"9814ecce14a3988eab86440ba9bc780b33aeb94d698cc4e465e79503494a7647"} Jul 02 22:04:38.756676 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.756656 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" event={"ID":"555df824-6fc1-4b40-a8dc-c6f5568d19c9","Type":"ContainerStarted","Data":"36b47934cd6daf359db6cfda6f47b03fa99ac6391ea2e7a628501d178e436d42"} Jul 02 22:04:38.758284 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.758264 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="3f1a8f33a08e098b91b531d078a781dc3b45b74fe285e04ab34afcfd1572c642" exitCode=0 Jul 02 22:04:38.758372 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.758349 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"3f1a8f33a08e098b91b531d078a781dc3b45b74fe285e04ab34afcfd1572c642"} Jul 02 22:04:38.761484 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"5da8bb823c65ea9df5d0088f1098a89e841050695dd163671b0c655d0a804d36"} Jul 02 22:04:38.761567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761524 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"35957f0fd336fde94075239bffce5d44cb108df10d41820751fb86c0afef47cd"} Jul 02 22:04:38.761567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"c051e4c4138e6bc2a27f4c5f1b0270de76332a266e787e9ec68d265b7a2abd4a"} Jul 02 22:04:38.761675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761591 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"022d3ef7def722b155425b61db2cff4d25a0b375c490dcb8ed592917def80710"} Jul 02 22:04:38.761675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761625 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"4f451ed84a269a930e51b04ad3b76b4945a77a02188b8ebf9604ab2811eef1a5"} Jul 02 22:04:38.761675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.761653 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"2d8b7df3b8ee4a53651af2806162426bf1202ed1f017bbfdf212efe16b511201"} Jul 02 22:04:38.762805 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.762780 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2qzt" event={"ID":"fb7ce651-2b47-47dd-8464-bce148a8b057","Type":"ContainerStarted","Data":"861602a3aeec2234f809abebb4fe84c0c8e9e2321b2e1abba480afeee6ea27ba"} Jul 02 22:04:38.764043 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.764023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zmxlq" event={"ID":"02517e0f-1525-477c-bee1-3bc04b0ceebb","Type":"ContainerStarted","Data":"c74786f040f1fe447d18b74d68f1debdd4813309cd815ff47ce96ff56b1e1e4e"} Jul 02 22:04:38.768419 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.768376 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fqvf7" podStartSLOduration=11.94836853 podStartE2EDuration="22.768363616s" totalImagesPullingTime="10.819995086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:38.767863319 +0000 UTC m=+23.752075400" watchObservedRunningTime="2026-07-02 22:04:38.768363616 +0000 UTC m=+23.752575696" Jul 02 22:04:38.783438 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.783391 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zmxlq" podStartSLOduration=11.952403884 podStartE2EDuration="22.783377417s" totalImagesPullingTime="10.830973533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:38.783245236 +0000 UTC m=+23.767457318" watchObservedRunningTime="2026-07-02 22:04:38.783377417 +0000 UTC m=+23.767589498" Jul 02 22:04:38.802256 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.802200 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5q49n" podStartSLOduration=3.842402637 podStartE2EDuration="22.802183725s" totalImagesPullingTime="18.959781088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:38.801718559 +0000 UTC m=+23.785930641" watchObservedRunningTime="2026-07-02 22:04:38.802183725 +0000 UTC m=+23.786395828" Jul 02 22:04:38.860635 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:38.860423 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h2qzt" podStartSLOduration=3.847486936 podStartE2EDuration="22.860409086s" totalImagesPullingTime="19.01292215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:38.860042396 +0000 UTC m=+23.844254477" watchObservedRunningTime="2026-07-02 22:04:38.860409086 +0000 UTC m=+23.844621167" Jul 02 22:04:39.503721 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.503692 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 22:04:39.520110 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.520002 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T22:04:39.503714063Z","UUID":"5dc5ee38-d2f2-4676-96c4-e5fcb1fae156","Handler":null,"Name":"","Endpoint":""} Jul 02 22:04:39.521800 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.521777 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 22:04:39.521923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.521822 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 22:04:39.595208 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.595176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:39.595404 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:39.595324 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:39.767912 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:39.767874 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" event={"ID":"555df824-6fc1-4b40-a8dc-c6f5568d19c9","Type":"ContainerStarted","Data":"6a52635fde6c2dc9e1294f8fd28a3e59be254275cb1bbfea71ff44cf0f5c72db"} Jul 02 22:04:40.595501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:40.595471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:40.595645 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:40.595477 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:40.595645 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:40.595581 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:40.595747 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:40.595688 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:40.774530 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:40.774488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"41eed7b745baede0485f6353d4f5ae23582cc34968f97e769e0f8290b23a90c2"} Jul 02 22:04:41.595423 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:41.595388 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:41.595683 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:41.595546 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:41.779623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:41.779588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" event={"ID":"555df824-6fc1-4b40-a8dc-c6f5568d19c9","Type":"ContainerStarted","Data":"d6ff5cc67b8fa0b944286e191d944d162b56049c68bc6ebcde716e46e6c48c81"} Jul 02 22:04:42.595637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:42.595596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:42.595815 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:42.595604 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:42.595815 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:42.595703 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:42.595815 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:42.595773 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:43.595472 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.595194 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:43.596203 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:43.595600 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:43.788927 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.788889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" event={"ID":"c25580bb-6095-4326-91af-eef364ec7f4d","Type":"ContainerStarted","Data":"f6a67230ac987bfe1e4bd63bed1922cf88cf842f328c6c9ec76f1769e879798e"} Jul 02 22:04:43.789238 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.789216 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:43.789369 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.789337 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:43.789424 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.789394 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:43.807229 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.807201 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:43.807417 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.807406 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:04:43.818599 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.818551 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kbfh" podStartSLOduration=14.357010434 podStartE2EDuration="27.818535682s" totalImagesPullingTime="13.461525248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:41.812524608 +0000 UTC m=+26.796736689" watchObservedRunningTime="2026-07-02 22:04:43.818535682 +0000 UTC m=+28.802747763" Jul 02 22:04:43.818874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:43.818850 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" podStartSLOduration=8.779095823 podStartE2EDuration="27.818844614s" totalImagesPullingTime="19.039748791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:43.818272093 +0000 UTC m=+28.802484189" watchObservedRunningTime="2026-07-02 22:04:43.818844614 +0000 UTC m=+28.803056695" Jul 02 22:04:44.595862 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:44.595827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:44.596370 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:44.595827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:44.596370 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:44.595953 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:44.596370 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:44.596006 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:45.149232 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.149200 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q55pk"] Jul 02 22:04:45.149383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.149354 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:45.149457 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:45.149433 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:45.152016 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.151988 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqlzp"] Jul 02 22:04:45.152132 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.152124 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:45.152267 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:45.152246 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:45.152632 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.152611 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7k68"] Jul 02 22:04:45.152707 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.152695 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:45.152792 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:45.152773 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:45.796413 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.796377 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="052d0c5907f96ae262e0a7e7f6a696f44a0e47028d79e36c3c79fb5e1af93144" exitCode=0 Jul 02 22:04:45.796780 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:45.796445 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"052d0c5907f96ae262e0a7e7f6a696f44a0e47028d79e36c3c79fb5e1af93144"} Jul 02 22:04:46.595288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:46.595242 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:46.595501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:46.595290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:46.595501 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:46.595418 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:46.595615 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:46.595548 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:46.805813 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:46.805769 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerStarted","Data":"a19d0c48cf6153dd2a80a61a1b5a86e669f7c851d332e47941bdc1d3cf65166f"} Jul 02 22:04:47.595052 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:47.594995 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:47.595259 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:47.595170 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:47.809670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:47.809633 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="a19d0c48cf6153dd2a80a61a1b5a86e669f7c851d332e47941bdc1d3cf65166f" exitCode=0 Jul 02 22:04:47.810149 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:47.809699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"a19d0c48cf6153dd2a80a61a1b5a86e669f7c851d332e47941bdc1d3cf65166f"} Jul 02 22:04:48.595230 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:48.595021 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:48.595423 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:48.595028 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:48.595423 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:48.595364 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:48.595423 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:48.595389 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:48.814642 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:48.814600 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="cb271f94dfb07d568023b97c06774ebf700eb38924281f1c15597d3334681d97" exitCode=0 Jul 02 22:04:48.814642 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:48.814640 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"cb271f94dfb07d568023b97c06774ebf700eb38924281f1c15597d3334681d97"} Jul 02 22:04:49.222324 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:49.222270 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:49.222465 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:49.222331 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:49.222465 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222439 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:49.222554 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222506 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:21.222488484 +0000 UTC m=+66.206700548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 22:04:49.222554 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222446 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 22:04:49.222554 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222551 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 22:04:49.222655 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222564 2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfx5z for pod openshift-network-diagnostics/network-check-target-n7k68: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:49.222655 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.222609 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z podName:d8ea5660-2949-4e35-a277-45282a22550f nodeName:}" failed. No retries permitted until 2026-07-02 22:05:21.222596761 +0000 UTC m=+66.206808820 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfx5z" (UniqueName: "kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z") pod "network-check-target-n7k68" (UID: "d8ea5660-2949-4e35-a277-45282a22550f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 22:04:49.323636 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:49.323596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:49.323803 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.323735 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:49.323803 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.323794 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret podName:0c995aee-a8c5-4d10-b49e-47da30a93d9b nodeName:}" failed. No retries permitted until 2026-07-02 22:05:21.323779732 +0000 UTC m=+66.307991809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret") pod "global-pull-secret-syncer-q55pk" (UID: "0c995aee-a8c5-4d10-b49e-47da30a93d9b") : object "kube-system"/"original-pull-secret" not registered Jul 02 22:04:49.595351 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:49.595314 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:49.595536 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:49.595458 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q55pk" podUID="0c995aee-a8c5-4d10-b49e-47da30a93d9b" Jul 02 22:04:50.595965 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.595886 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:50.595965 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.595934 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:50.596403 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:50.596006 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7k68" podUID="d8ea5660-2949-4e35-a277-45282a22550f" Jul 02 22:04:50.596403 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:50.596136 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqlzp" podUID="4a4ca393-cf0e-4866-b73a-76ad7790a42d" Jul 02 22:04:50.789696 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.789667 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-202.ec2.internal" event="NodeReady" Jul 02 22:04:50.789844 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.789786 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 22:04:50.830213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.830174 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:04:50.834269 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.834248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.837546 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.837524 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 22:04:50.837658 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.837606 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qk2ld\"" Jul 02 22:04:50.837658 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.837635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 22:04:50.837818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.837802 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 22:04:50.841418 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.841398 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tpg8k"] Jul 02 22:04:50.845474 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.845454 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:50.845559 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.845520 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 22:04:50.847092 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.847025 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fxxn5"] Jul 02 22:04:50.850728 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.850700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:50.850971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.850892 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 22:04:50.850971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.850943 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:04:50.851450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.851430 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8wxkj\"" Jul 02 22:04:50.851552 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.851439 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 22:04:50.853377 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.853326 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 22:04:50.853590 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.853571 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 22:04:50.853704 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.853619 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 22:04:50.853766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.853752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lwzc2\"" Jul 02 22:04:50.862493 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.862467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxxn5"] Jul 02 22:04:50.866902 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.866875 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tpg8k"] Jul 02 22:04:50.935861 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.935831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.935861 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.935864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbf4\" (UniqueName: \"kubernetes.io/projected/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-kube-api-access-ksbf4\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:50.936067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.935892 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.935926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:50.936067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.935953 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kzx6\" (UniqueName: \"kubernetes.io/projected/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-kube-api-access-9kzx6\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:50.936067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-tmp-dir\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936107 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936223 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-config-volume\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-cert\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:50.936374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.936318 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49twd\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:50.970192 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:50.970157 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:51.037188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037157 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-config-volume\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.037188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-cert\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:51.037468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49twd\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbf4\" (UniqueName: \"kubernetes.io/projected/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-kube-api-access-ksbf4\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kzx6\" (UniqueName: \"kubernetes.io/projected/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-kube-api-access-9kzx6\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037592 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-tmp-dir\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.037616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037653 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037895 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.037938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037909 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-config-volume\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.038105 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.037942 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:04:51.038869 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.038721 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-tmp-dir\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.042624 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.039007 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:04:51.042624 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.039028 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:04:51.042624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.037955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.042624 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.039074 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:51.539038391 +0000 UTC m=+36.523250450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:04:51.042624 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.039100 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:04:51.53908795 +0000 UTC m=+36.523300012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:04:51.043167 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.043119 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.043342 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.043165 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.043745 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.043210 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-cert\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:51.043745 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.043229 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.048743 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.048717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.048854 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.048834 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kzx6\" (UniqueName: \"kubernetes.io/projected/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-kube-api-access-9kzx6\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.049265 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.049239 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49twd\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.049961 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.049902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbf4\" (UniqueName: \"kubernetes.io/projected/ac9ca684-7062-49c0-bbaf-5dd463e3ecfb-kube-api-access-ksbf4\") pod \"ingress-canary-fxxn5\" (UID: \"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb\") " pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:51.063530 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.063500 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:51.162787 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.162701 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxxn5" Jul 02 22:04:51.308343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.308290 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxxn5"] Jul 02 22:04:51.314491 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:04:51.314462 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9ca684_7062_49c0_bbaf_5dd463e3ecfb.slice/crio-2ad11d56fe2b2212e889e8e72da047250dd60423bf0ac124d472478bbaecdec2 WatchSource:0}: Error finding container 2ad11d56fe2b2212e889e8e72da047250dd60423bf0ac124d472478bbaecdec2: Status 404 returned error can't find the container with id 2ad11d56fe2b2212e889e8e72da047250dd60423bf0ac124d472478bbaecdec2 Jul 02 22:04:51.541617 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.541583 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:51.541801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.541643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:51.541801 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.541778 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:04:51.541940 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.541810 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:04:51.541940 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.541847 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:52.541826799 +0000 UTC m=+37.526038872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:04:51.541940 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.541848 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:04:51.541940 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:51.541887 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:04:52.54187803 +0000 UTC m=+37.526090092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:04:51.595704 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.595666 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:04:51.599120 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.599093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 22:04:51.823925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:51.823658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxxn5" event={"ID":"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb","Type":"ContainerStarted","Data":"2ad11d56fe2b2212e889e8e72da047250dd60423bf0ac124d472478bbaecdec2"} Jul 02 22:04:52.550917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.550876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:52.551106 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.550988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:52.551106 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:52.551045 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:04:52.551106 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:52.551074 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:04:52.551106 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:52.551085 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:04:52.551281 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:52.551116 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:54.551096938 +0000 UTC m=+39.535309000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:04:52.551281 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:52.551132 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:04:54.551125895 +0000 UTC m=+39.535337954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:04:52.595837 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.595799 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:04:52.596000 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.595812 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:04:52.598951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.598930 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ddmbm\"" Jul 02 22:04:52.598951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.598946 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 22:04:52.600070 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.600044 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 22:04:52.600070 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.600060 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 22:04:52.600537 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:52.600226 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lnnqk\"" Jul 02 22:04:54.570915 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:54.570881 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:54.570996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:54.571068 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:54.571124 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:54.571141 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:54.571147 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:04:58.571124618 +0000 UTC m=+43.555336690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:04:54.571350 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:54.571184 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:04:58.571173058 +0000 UTC m=+43.555385131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:04:55.837218 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:55.837186 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="eeac4ef5ce00a6085189a9f455884ee9ec52095d4c002c75f9a458d716066663" exitCode=0 Jul 02 22:04:55.837768 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:55.837270 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"eeac4ef5ce00a6085189a9f455884ee9ec52095d4c002c75f9a458d716066663"} Jul 02 22:04:55.838834 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:55.838773 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxxn5" event={"ID":"ac9ca684-7062-49c0-bbaf-5dd463e3ecfb","Type":"ContainerStarted","Data":"753b650ddf5a9c67e0397088d9a1b12f71c4b674161033cff624ea5656b43466"} Jul 02 22:04:55.877287 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:55.877237 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fxxn5" podStartSLOduration=2.46753942 podStartE2EDuration="5.877216481s" totalImagesPullingTime="3.409677061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:55.876513741 +0000 UTC m=+40.860725834" watchObservedRunningTime="2026-07-02 22:04:55.877216481 +0000 UTC m=+40.861428564" Jul 02 22:04:56.084738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:56.084545 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:56.184954 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:56.184917 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:56.843846 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:56.843815 2583 generic.go:356] "Generic (PLEG): container finished" podID="f8c728c9-da49-4fba-9184-e4c93b0b197e" containerID="19b3d8d3c36b748adccdbfe4d15e53694c83e3857e3763f04b4bcbb7a24b53ea" exitCode=0 Jul 02 22:04:56.844283 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:56.843892 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerDied","Data":"19b3d8d3c36b748adccdbfe4d15e53694c83e3857e3763f04b4bcbb7a24b53ea"} Jul 02 22:04:57.849345 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:57.849295 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llrgq" event={"ID":"f8c728c9-da49-4fba-9184-e4c93b0b197e","Type":"ContainerStarted","Data":"adb4815faa6857514c6f17b44bfebf7bb22ff7f4b56ded2bf3793b4df21edd8f"} Jul 02 22:04:57.877191 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:57.877140 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-llrgq" podStartSLOduration=17.211070449 podStartE2EDuration="41.877124871s" totalImagesPullingTime="24.666054422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:57.873856607 +0000 UTC m=+42.858068689" watchObservedRunningTime="2026-07-02 22:04:57.877124871 +0000 UTC m=+42.861336951" Jul 02 22:04:57.995708 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:57.995665 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:58.051391 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:58.051352 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:58.601554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:58.601507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:04:58.601733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:58.601622 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:04:58.601733 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:58.601657 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:04:58.601733 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:58.601724 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:05:06.601707622 +0000 UTC m=+51.585919685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:04:58.601910 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:58.601745 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:04:58.601910 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:58.601758 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:04:58.601910 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:04:58.601808 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:05:06.60179251 +0000 UTC m=+51.586004582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:04:58.864250 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:58.864129 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:58.965949 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:58.965919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:59.026679 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:59.026638 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:59.877059 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:59.877028 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:59.975850 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:04:59.975817 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:01.844171 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:01.844125 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:03.680283 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:03.680254 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:03.732109 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:03.732070 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:06.658900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:06.658856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:06.658916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:06.659019 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:06.659034 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:06.659092 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:05:22.659077072 +0000 UTC m=+67.643289137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:06.659093 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:05:06.659280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:06.659152 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:05:22.659133858 +0000 UTC m=+67.643345920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:05:08.304823 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:08.304785 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:08.403525 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:08.403480 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:09.307930 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:09.307893 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:09.403894 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:09.403859 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:10.466009 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.465983 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9"] Jul 02 22:05:10.467677 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.467661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.469318 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.469286 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6"] Jul 02 22:05:10.470329 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.470294 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-spfkp\"" Jul 02 22:05:10.470447 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.470437 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 22:05:10.470797 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.470783 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" Jul 02 22:05:10.471483 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.471470 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 22:05:10.473781 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.473759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.473887 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.473790 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-f79cc\"" Jul 02 22:05:10.474110 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.474089 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.481753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.481698 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9"] Jul 02 22:05:10.482793 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.482769 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6"] Jul 02 22:05:10.487093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.487067 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49"] Jul 02 22:05:10.487495 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.487476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/518c9c14-f70e-45a0-a566-80b717650204-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.487589 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.487546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcdf8\" (UniqueName: \"kubernetes.io/projected/b5c221fd-0fd5-40ad-9562-a5af0216d438-kube-api-access-jcdf8\") pod \"volume-data-source-validator-5d456fdbcd-8f7h6\" (UID: \"b5c221fd-0fd5-40ad-9562-a5af0216d438\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" Jul 02 22:05:10.487649 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.487627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.488889 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.488870 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d48d4fd89-qjr8q"] Jul 02 22:05:10.489019 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.489002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.491069 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.491049 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.491844 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.491826 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 02 22:05:10.491925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.491848 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.492006 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.491989 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.492062 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.492048 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5sv64\"" Jul 02 22:05:10.493529 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.493513 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 02 22:05:10.493617 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.493548 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 02 22:05:10.493617 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.493554 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n4fqx\"" Jul 02 22:05:10.493822 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.493807 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.494364 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.494343 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.494489 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.494389 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 02 22:05:10.494489 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.494410 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 02 22:05:10.495667 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.495649 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs"] Jul 02 22:05:10.497634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.497617 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" Jul 02 22:05:10.500585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.500554 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-99sdd\"" Jul 02 22:05:10.501261 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.501241 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49"] Jul 02 22:05:10.508726 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.508701 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d48d4fd89-qjr8q"] Jul 02 22:05:10.510288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.510266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs"] Jul 02 22:05:10.571335 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.571288 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs"] Jul 02 22:05:10.573275 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.573258 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.576408 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.576349 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.576772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.576737 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 02 22:05:10.576867 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.576743 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.577656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.577634 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-29zj6\"" Jul 02 22:05:10.577793 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.577742 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 02 22:05:10.578185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.578168 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx"] Jul 02 22:05:10.579835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.579814 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-srrnc"] Jul 02 22:05:10.579934 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.579902 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.581600 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.581585 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gd55z"] Jul 02 22:05:10.581716 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.581698 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.582480 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.582460 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 22:05:10.582480 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.582463 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g2wqj\"" Jul 02 22:05:10.582641 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.582470 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.583249 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.583234 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb"] Jul 02 22:05:10.583370 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.583355 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.584197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.584171 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.584287 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.584173 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 22:05:10.584805 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.584785 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 02 22:05:10.585035 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.585017 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.585396 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.585377 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.585518 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.585504 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 02 22:05:10.585591 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.585569 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.586589 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.586558 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-795bc\"" Jul 02 22:05:10.592388 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.591514 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 22:05:10.592388 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.591888 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.592750 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.592678 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 02 22:05:10.592750 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.592738 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 02 22:05:10.592940 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.592921 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.594259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.593259 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 22:05:10.594259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.594084 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dlqzd\"" Jul 02 22:05:10.594681 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.594662 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wvqbd\"" Jul 02 22:05:10.595203 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.595181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.595288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.595238 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.595492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.595385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d469a43-f522-4f7c-b597-2ae7c3bff1db-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.595492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.595452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.595944 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.595922 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 22:05:10.597543 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.597521 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.597663 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.597596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.597736 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.597697 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b86f8bb3-61ae-4735-97b2-9d11dec6e535-config\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.598559 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.598539 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:05:10.599187 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/855a99f7-2255-47c6-831b-8b9eebbfc19c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.599345 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs"] Jul 02 22:05:10.599532 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/518c9c14-f70e-45a0-a566-80b717650204-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.599672 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599651 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-trusted-ca\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.599794 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599774 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-tmp\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.599930 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.599908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbvnq\" (UniqueName: \"kubernetes.io/projected/b86f8bb3-61ae-4735-97b2-9d11dec6e535-kube-api-access-zbvnq\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.600050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-snapshots\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.600222 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-serving-cert\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.600353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600246 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl45p\" (UniqueName: \"kubernetes.io/projected/9d469a43-f522-4f7c-b597-2ae7c3bff1db-kube-api-access-jl45p\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.600353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rssnp\" (UniqueName: \"kubernetes.io/projected/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-kube-api-access-rssnp\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.600588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600377 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgwz\" (UniqueName: \"kubernetes.io/projected/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-kube-api-access-frgwz\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.600588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-844dh\" (UniqueName: \"kubernetes.io/projected/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-kube-api-access-844dh\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.600588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600502 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32d31b2-74db-42d9-adfe-24d20e516268-serving-cert\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.600588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t5r8\" (UniqueName: \"kubernetes.io/projected/c32d31b2-74db-42d9-adfe-24d20e516268-kube-api-access-7t5r8\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600583 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600629 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tw5\" (UniqueName: \"kubernetes.io/projected/9048ce8d-95ab-4fca-9890-bc5b7af4eb2d-kube-api-access-z2tw5\") pod \"network-check-source-989f8b8dc-m6tzs\" (UID: \"9048ce8d-95ab-4fca-9890-bc5b7af4eb2d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-config\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600734 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcdf8\" (UniqueName: \"kubernetes.io/projected/b5c221fd-0fd5-40ad-9562-a5af0216d438-kube-api-access-jcdf8\") pod \"volume-data-source-validator-5d456fdbcd-8f7h6\" (UID: \"b5c221fd-0fd5-40ad-9562-a5af0216d438\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d469a43-f522-4f7c-b597-2ae7c3bff1db-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b86f8bb3-61ae-4735-97b2-9d11dec6e535-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hwns\" (UniqueName: \"kubernetes.io/projected/855a99f7-2255-47c6-831b-8b9eebbfc19c-kube-api-access-4hwns\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-default-certificate\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.600979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-stats-auth\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.601081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.601041 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 02 22:05:10.601737 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.601373 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gd55z"] Jul 02 22:05:10.601737 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.601463 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/518c9c14-f70e-45a0-a566-80b717650204-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:10.601737 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.601521 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:10.601737 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.601593 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:11.101574246 +0000 UTC m=+56.085786307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:10.602438 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.602420 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb"] Jul 02 22:05:10.603372 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.603353 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx"] Jul 02 22:05:10.603554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.603536 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 02 22:05:10.623529 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.623501 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcdf8\" (UniqueName: \"kubernetes.io/projected/b5c221fd-0fd5-40ad-9562-a5af0216d438-kube-api-access-jcdf8\") pod \"volume-data-source-validator-5d456fdbcd-8f7h6\" (UID: \"b5c221fd-0fd5-40ad-9562-a5af0216d438\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" Jul 02 22:05:10.630325 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.630284 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-srrnc"] Jul 02 22:05:10.668617 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.668577 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:10.701477 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32d31b2-74db-42d9-adfe-24d20e516268-serving-cert\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.701477 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t5r8\" (UniqueName: \"kubernetes.io/projected/c32d31b2-74db-42d9-adfe-24d20e516268-kube-api-access-7t5r8\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tw5\" (UniqueName: \"kubernetes.io/projected/9048ce8d-95ab-4fca-9890-bc5b7af4eb2d-kube-api-access-z2tw5\") pod \"network-check-source-989f8b8dc-m6tzs\" (UID: \"9048ce8d-95ab-4fca-9890-bc5b7af4eb2d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-config\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701578 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d469a43-f522-4f7c-b597-2ae7c3bff1db-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b86f8bb3-61ae-4735-97b2-9d11dec6e535-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701650 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hwns\" (UniqueName: \"kubernetes.io/projected/855a99f7-2255-47c6-831b-8b9eebbfc19c-kube-api-access-4hwns\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-default-certificate\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.701733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-stats-auth\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701828 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d469a43-f522-4f7c-b597-2ae7c3bff1db-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.701986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b86f8bb3-61ae-4735-97b2-9d11dec6e535-config\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/855a99f7-2255-47c6-831b-8b9eebbfc19c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-trusted-ca\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.702125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-tmp\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbvnq\" (UniqueName: \"kubernetes.io/projected/b86f8bb3-61ae-4735-97b2-9d11dec6e535-kube-api-access-zbvnq\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702157 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-snapshots\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702183 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d469a43-f522-4f7c-b597-2ae7c3bff1db-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-serving-cert\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl45p\" (UniqueName: \"kubernetes.io/projected/9d469a43-f522-4f7c-b597-2ae7c3bff1db-kube-api-access-jl45p\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rssnp\" (UniqueName: \"kubernetes.io/projected/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-kube-api-access-rssnp\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgwz\" (UniqueName: \"kubernetes.io/projected/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-kube-api-access-frgwz\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.702628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702346 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-844dh\" (UniqueName: \"kubernetes.io/projected/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-kube-api-access-844dh\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.703022 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b86f8bb3-61ae-4735-97b2-9d11dec6e535-config\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.703022 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.702910 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.703130 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.703099 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-config\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.703267 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/855a99f7-2255-47c6-831b-8b9eebbfc19c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.703286 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.703374 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:11.203356893 +0000 UTC m=+56.187568954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.703698 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-trusted-ca\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.703717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32d31b2-74db-42d9-adfe-24d20e516268-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.703869 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:11.203853968 +0000 UTC m=+56.188066031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.703986 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.704037 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.704047 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:11.204032094 +0000 UTC m=+56.188244175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.703986 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-snapshots\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.704093 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:10.704094 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:11.204078302 +0000 UTC m=+56.188290368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:10.704900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.704363 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32d31b2-74db-42d9-adfe-24d20e516268-tmp\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.704900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.704660 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-default-certificate\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.704900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.704715 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32d31b2-74db-42d9-adfe-24d20e516268-serving-cert\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.705050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.705009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b86f8bb3-61ae-4735-97b2-9d11dec6e535-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.705216 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.705196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d469a43-f522-4f7c-b597-2ae7c3bff1db-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.706078 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.706063 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-stats-auth\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.706515 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.706500 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-serving-cert\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.712815 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.712768 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t5r8\" (UniqueName: \"kubernetes.io/projected/c32d31b2-74db-42d9-adfe-24d20e516268-kube-api-access-7t5r8\") pod \"insights-operator-5c847dcfc4-srrnc\" (UID: \"c32d31b2-74db-42d9-adfe-24d20e516268\") " pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.714147 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.714116 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl45p\" (UniqueName: \"kubernetes.io/projected/9d469a43-f522-4f7c-b597-2ae7c3bff1db-kube-api-access-jl45p\") pod \"kube-storage-version-migrator-operator-846bf668f7-gcwlb\" (UID: \"9d469a43-f522-4f7c-b597-2ae7c3bff1db\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.715042 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.715022 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tw5\" (UniqueName: \"kubernetes.io/projected/9048ce8d-95ab-4fca-9890-bc5b7af4eb2d-kube-api-access-z2tw5\") pod \"network-check-source-989f8b8dc-m6tzs\" (UID: \"9048ce8d-95ab-4fca-9890-bc5b7af4eb2d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" Jul 02 22:05:10.715580 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.715558 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-844dh\" (UniqueName: \"kubernetes.io/projected/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-kube-api-access-844dh\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:10.715666 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.715620 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgwz\" (UniqueName: \"kubernetes.io/projected/3ae65cac-b4f8-4b54-944b-b6fd61f251c0-kube-api-access-frgwz\") pod \"console-operator-68b757fb67-gd55z\" (UID: \"3ae65cac-b4f8-4b54-944b-b6fd61f251c0\") " pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.715974 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.715955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbvnq\" (UniqueName: \"kubernetes.io/projected/b86f8bb3-61ae-4735-97b2-9d11dec6e535-kube-api-access-zbvnq\") pod \"service-ca-operator-5d46d9ffd4-7wbrs\" (UID: \"b86f8bb3-61ae-4735-97b2-9d11dec6e535\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.716389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.716345 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rssnp\" (UniqueName: \"kubernetes.io/projected/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-kube-api-access-rssnp\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:10.716443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.716427 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hwns\" (UniqueName: \"kubernetes.io/projected/855a99f7-2255-47c6-831b-8b9eebbfc19c-kube-api-access-4hwns\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:10.756990 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.756954 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:10.785182 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.785147 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" Jul 02 22:05:10.812779 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.812744 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" Jul 02 22:05:10.821059 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.821009 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:10.883953 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.883846 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" Jul 02 22:05:10.902888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.902859 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" Jul 02 22:05:10.909179 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.909144 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:10.914957 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.914938 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" Jul 02 22:05:10.926162 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.926130 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6"] Jul 02 22:05:10.931366 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:10.931333 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c221fd_0fd5_40ad_9562_a5af0216d438.slice/crio-ab9b19d4540f345e0eee2577c1e0cdaab12920342e18f451ca1730b9bc7de4c5 WatchSource:0}: Error finding container ab9b19d4540f345e0eee2577c1e0cdaab12920342e18f451ca1730b9bc7de4c5: Status 404 returned error can't find the container with id ab9b19d4540f345e0eee2577c1e0cdaab12920342e18f451ca1730b9bc7de4c5 Jul 02 22:05:10.954772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:10.954728 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs"] Jul 02 22:05:10.960170 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:10.960102 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9048ce8d_95ab_4fca_9890_bc5b7af4eb2d.slice/crio-f075048a415457e3976c71ffe0ceeeb6fad530cb821a25402e1e3ebe39f437d2 WatchSource:0}: Error finding container f075048a415457e3976c71ffe0ceeeb6fad530cb821a25402e1e3ebe39f437d2: Status 404 returned error can't find the container with id f075048a415457e3976c71ffe0ceeeb6fad530cb821a25402e1e3ebe39f437d2 Jul 02 22:05:11.065586 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.065052 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs"] Jul 02 22:05:11.068463 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:11.068423 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86f8bb3_61ae_4735_97b2_9d11dec6e535.slice/crio-87041b35216486cad804c3ba4a1b35685daa9794f25fde6d4c66814a34c839d2 WatchSource:0}: Error finding container 87041b35216486cad804c3ba4a1b35685daa9794f25fde6d4c66814a34c839d2: Status 404 returned error can't find the container with id 87041b35216486cad804c3ba4a1b35685daa9794f25fde6d4c66814a34c839d2 Jul 02 22:05:11.091778 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.091730 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-srrnc"] Jul 02 22:05:11.094413 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:11.094384 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32d31b2_74db_42d9_adfe_24d20e516268.slice/crio-d977e66e24f93b02893f77c153adec50cd6d85cfde70d8e2444fe645432c6aba WatchSource:0}: Error finding container d977e66e24f93b02893f77c153adec50cd6d85cfde70d8e2444fe645432c6aba: Status 404 returned error can't find the container with id d977e66e24f93b02893f77c153adec50cd6d85cfde70d8e2444fe645432c6aba Jul 02 22:05:11.105915 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.105877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:11.106060 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.106032 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:11.106123 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.106107 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:12.106086368 +0000 UTC m=+57.090298427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:11.107163 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.107132 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb"] Jul 02 22:05:11.108060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.108039 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-gd55z"] Jul 02 22:05:11.111486 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:11.111455 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d469a43_f522_4f7c_b597_2ae7c3bff1db.slice/crio-42026b8a909bd1bd601c43c49b1619618dd72078fc49801b528beb14f3ad83b8 WatchSource:0}: Error finding container 42026b8a909bd1bd601c43c49b1619618dd72078fc49801b528beb14f3ad83b8: Status 404 returned error can't find the container with id 42026b8a909bd1bd601c43c49b1619618dd72078fc49801b528beb14f3ad83b8 Jul 02 22:05:11.111955 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:11.111931 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae65cac_b4f8_4b54_944b_b6fd61f251c0.slice/crio-7445a8d06709268d619c22d20f1c751e1f58df5fda1963af475123258fb6e0a3 WatchSource:0}: Error finding container 7445a8d06709268d619c22d20f1c751e1f58df5fda1963af475123258fb6e0a3: Status 404 returned error can't find the container with id 7445a8d06709268d619c22d20f1c751e1f58df5fda1963af475123258fb6e0a3 Jul 02 22:05:11.206430 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.206399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:11.206605 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.206452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:11.206605 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.206525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:11.206605 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206538 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:11.206605 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.206569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:11.206605 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206597 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:12.206580232 +0000 UTC m=+57.190792329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:11.206875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206619 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:11.206875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206655 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:11.206875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206676 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:12.20666036 +0000 UTC m=+57.190872419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:11.206875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206698 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:12.2066874 +0000 UTC m=+57.190899459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:11.206875 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:11.206715 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:12.206705118 +0000 UTC m=+57.190917180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:11.207107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.207020 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:11.306256 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.306154 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:11.889145 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.889107 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" event={"ID":"b86f8bb3-61ae-4735-97b2-9d11dec6e535","Type":"ContainerStarted","Data":"87041b35216486cad804c3ba4a1b35685daa9794f25fde6d4c66814a34c839d2"} Jul 02 22:05:11.890753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.890695 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" event={"ID":"9d469a43-f522-4f7c-b597-2ae7c3bff1db","Type":"ContainerStarted","Data":"42026b8a909bd1bd601c43c49b1619618dd72078fc49801b528beb14f3ad83b8"} Jul 02 22:05:11.892260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.892221 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" event={"ID":"c32d31b2-74db-42d9-adfe-24d20e516268","Type":"ContainerStarted","Data":"d977e66e24f93b02893f77c153adec50cd6d85cfde70d8e2444fe645432c6aba"} Jul 02 22:05:11.893702 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.893617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" event={"ID":"b5c221fd-0fd5-40ad-9562-a5af0216d438","Type":"ContainerStarted","Data":"ab9b19d4540f345e0eee2577c1e0cdaab12920342e18f451ca1730b9bc7de4c5"} Jul 02 22:05:11.895501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.895465 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerStarted","Data":"7445a8d06709268d619c22d20f1c751e1f58df5fda1963af475123258fb6e0a3"} Jul 02 22:05:11.899755 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:11.899725 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" event={"ID":"9048ce8d-95ab-4fca-9890-bc5b7af4eb2d","Type":"ContainerStarted","Data":"f075048a415457e3976c71ffe0ceeeb6fad530cb821a25402e1e3ebe39f437d2"} Jul 02 22:05:12.116249 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.116207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:12.116526 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.116508 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:12.116606 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.116579 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:14.116559718 +0000 UTC m=+59.100771780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.217103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.217151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.217208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.217244 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217258 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217314 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217350 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:14.217327431 +0000 UTC m=+59.201539514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217370 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:14.217361003 +0000 UTC m=+59.201573066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217369 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217385 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:14.217377577 +0000 UTC m=+59.201589640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:12.217442 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:12.217416 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:14.217401331 +0000 UTC m=+59.201613401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:12.906590 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.906101 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" event={"ID":"b5c221fd-0fd5-40ad-9562-a5af0216d438","Type":"ContainerStarted","Data":"adf9434b3057ad32386910f4175343be6290523d3e64b09079c1d44501cb839a"} Jul 02 22:05:12.926955 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:12.926888 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8f7h6" podStartSLOduration=1.38376879 podStartE2EDuration="2.926862625s" totalImagesPullingTime="1.543093835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:12.92575071 +0000 UTC m=+57.909962792" watchObservedRunningTime="2026-07-02 22:05:12.926862625 +0000 UTC m=+57.911074716" Jul 02 22:05:13.040138 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:13.040107 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:13.110259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:13.110222 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:14.136107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:14.136016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:14.141342 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.136722 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:14.141342 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.136818 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.136795908 +0000 UTC m=+63.121007967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:14.237036 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:14.236997 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:14.237036 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:14.237041 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:14.237090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:14.237121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237197 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237240 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237284 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.237261345 +0000 UTC m=+63.221473416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237324 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.237295034 +0000 UTC m=+63.221507098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237346 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.237337348 +0000 UTC m=+63.221549407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237348 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:14.237446 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:14.237396 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.237381956 +0000 UTC m=+63.221594015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:15.817917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:15.817883 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpms" Jul 02 22:05:15.975658 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:15.975616 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:16.077769 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.077674 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:16.548779 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.548747 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:16.648407 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.648377 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:16.919923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.919887 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" event={"ID":"b86f8bb3-61ae-4735-97b2-9d11dec6e535","Type":"ContainerStarted","Data":"42abe870d15b163cbddb946e6b3b214592c999997ca8de52144bf2f2d4969797"} Jul 02 22:05:16.922203 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.922161 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" event={"ID":"9d469a43-f522-4f7c-b597-2ae7c3bff1db","Type":"ContainerStarted","Data":"8456ba213e893097844f59492d481b6e20f79f27b1e80da0fd44ea4f53c94507"} Jul 02 22:05:16.924802 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.924766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" event={"ID":"c32d31b2-74db-42d9-adfe-24d20e516268","Type":"ContainerStarted","Data":"4113fc28ce597cee6847e4915fe57860552a7dd3253454cce0637115eeae0ba6"} Jul 02 22:05:16.927022 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.926991 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerStarted","Data":"d6dd69e4e328fb7259f429c6983d573a397e815e8c1a06ec487c5308807e56d4"} Jul 02 22:05:16.927565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.927544 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:16.928859 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.928786 2583 patch_prober.go:28] interesting pod/console-operator-68b757fb67-gd55z container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jul 02 22:05:16.928859 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.928841 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused" Jul 02 22:05:16.929237 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.929209 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" event={"ID":"9048ce8d-95ab-4fca-9890-bc5b7af4eb2d","Type":"ContainerStarted","Data":"aebe455a40180c8a573f725e401ffa5e04ec4ae38798533bacd24df6ea111340"} Jul 02 22:05:16.940855 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.940754 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" podStartSLOduration=1.343863014 podStartE2EDuration="6.940732218s" totalImagesPullingTime="5.596869204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:16.940381222 +0000 UTC m=+61.924593303" watchObservedRunningTime="2026-07-02 22:05:16.940732218 +0000 UTC m=+61.924944295" Jul 02 22:05:16.995338 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.994857 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m6tzs" podStartSLOduration=1.279302342 podStartE2EDuration="6.994841699s" totalImagesPullingTime="5.715539357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:16.959802143 +0000 UTC m=+61.944014224" watchObservedRunningTime="2026-07-02 22:05:16.994841699 +0000 UTC m=+61.979053780" Jul 02 22:05:16.995338 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:16.995095 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podStartSLOduration=1.441614025 podStartE2EDuration="6.995089305s" totalImagesPullingTime="5.55347528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:16.994055392 +0000 UTC m=+61.978267474" watchObservedRunningTime="2026-07-02 22:05:16.995089305 +0000 UTC m=+61.979301386" Jul 02 22:05:17.017404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.017335 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" podStartSLOduration=1.446636102 podStartE2EDuration="7.017289585s" totalImagesPullingTime="5.570653483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:17.016462518 +0000 UTC m=+62.000674611" watchObservedRunningTime="2026-07-02 22:05:17.017289585 +0000 UTC m=+62.001501669" Jul 02 22:05:17.037802 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.037739 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" podStartSLOduration=1.483437887 podStartE2EDuration="7.037717187s" totalImagesPullingTime="5.5542793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:17.035509708 +0000 UTC m=+62.019721790" watchObservedRunningTime="2026-07-02 22:05:17.037717187 +0000 UTC m=+62.021929268" Jul 02 22:05:17.639908 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.639871 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z5vzl"] Jul 02 22:05:17.642632 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.642613 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.645641 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.645608 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsr6s\"" Jul 02 22:05:17.645780 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.645731 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 22:05:17.646900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.646877 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 22:05:17.655122 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.655090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5vzl"] Jul 02 22:05:17.682055 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.682018 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:17.741316 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.741258 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:17.771311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.771272 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.771479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.771350 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ch6\" (UniqueName: \"kubernetes.io/projected/0344693d-b473-4240-a043-8a5e609a6793-kube-api-access-f8ch6\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.771479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.771386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0344693d-b473-4240-a043-8a5e609a6793-data-volume\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.771571 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.771511 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.771640 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.771624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0344693d-b473-4240-a043-8a5e609a6793-crio-socket\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872396 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0344693d-b473-4240-a043-8a5e609a6793-crio-socket\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:17.872497 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ch6\" (UniqueName: \"kubernetes.io/projected/0344693d-b473-4240-a043-8a5e609a6793-kube-api-access-f8ch6\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0344693d-b473-4240-a043-8a5e609a6793-data-volume\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872582 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:17.872574 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.372552665 +0000 UTC m=+63.356764735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : configmap "kube-rbac-proxy" not found Jul 02 22:05:17.872885 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872572 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0344693d-b473-4240-a043-8a5e609a6793-crio-socket\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.872885 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:17.872708 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:17.872885 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:17.872742 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:18.372731778 +0000 UTC m=+63.356943840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:17.873035 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.872911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0344693d-b473-4240-a043-8a5e609a6793-data-volume\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.883791 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.883754 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ch6\" (UniqueName: \"kubernetes.io/projected/0344693d-b473-4240-a043-8a5e609a6793-kube-api-access-f8ch6\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:17.934698 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.934671 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/0.log" Jul 02 22:05:17.935162 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.934712 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" containerID="d6dd69e4e328fb7259f429c6983d573a397e815e8c1a06ec487c5308807e56d4" exitCode=255 Jul 02 22:05:17.935162 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.934794 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerDied","Data":"d6dd69e4e328fb7259f429c6983d573a397e815e8c1a06ec487c5308807e56d4"} Jul 02 22:05:17.935162 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.935099 2583 scope.go:117] "RemoveContainer" containerID="d6dd69e4e328fb7259f429c6983d573a397e815e8c1a06ec487c5308807e56d4" Jul 02 22:05:17.956133 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:17.956094 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:18.021452 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.021412 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:18.176640 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.176600 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:18.176813 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.176774 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:18.176884 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.176872 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:26.176850089 +0000 UTC m=+71.161062154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:18.277862 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.277765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:18.277862 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.277819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.277875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.277904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.277950 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278000 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278041 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278048 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:26.278026619 +0000 UTC m=+71.262238683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278067 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:26.278058239 +0000 UTC m=+71.262270304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:18.278087 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278082 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:26.278073897 +0000 UTC m=+71.262285976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:18.278447 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.278097 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:26.278089289 +0000 UTC m=+71.262301352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:18.378900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.378865 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:18.379107 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.379059 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:18.379184 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.379126 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 22:05:18.379184 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.379138 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:19.37911786 +0000 UTC m=+64.363329932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:18.379184 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.379060 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:18.379329 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.379179 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:19.379162146 +0000 UTC m=+64.363374219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : configmap "kube-rbac-proxy" not found Jul 02 22:05:18.388481 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.388446 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:18.468005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.467973 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:18.533841 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.533738 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:18.943098 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943068 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/1.log" Jul 02 22:05:18.943589 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943568 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/0.log" Jul 02 22:05:18.943633 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943617 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" exitCode=255 Jul 02 22:05:18.943698 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943681 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerDied","Data":"151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050"} Jul 02 22:05:18.943787 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943774 2583 scope.go:117] "RemoveContainer" containerID="d6dd69e4e328fb7259f429c6983d573a397e815e8c1a06ec487c5308807e56d4" Jul 02 22:05:18.944017 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:18.943994 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:18.944200 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:18.944180 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:19.388350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.388233 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:19.388518 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.388366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:19.388518 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:19.388425 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:19.388518 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:19.388448 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 22:05:19.388518 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:19.388506 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:21.3884902 +0000 UTC m=+66.372702260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:19.388676 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:19.388522 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:21.388515178 +0000 UTC m=+66.372727237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : configmap "kube-rbac-proxy" not found Jul 02 22:05:19.409813 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.409783 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zmxlq_02517e0f-1525-477c-bee1-3bc04b0ceebb/dns-node-resolver/0.log" Jul 02 22:05:19.582190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.582153 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:19.672640 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.672555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:19.948802 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.948775 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/1.log" Jul 02 22:05:19.949180 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.949117 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:19.949280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:19.949261 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:19.998797 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:19.998760 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:20.056078 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.056041 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:20.277499 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.277391 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:20.377819 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.377787 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:20.409736 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.409711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fqvf7_69304590-8e37-4dc2-a16a-b8e8ea9208a5/node-ca/0.log" Jul 02 22:05:20.809227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.809192 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxxn5_ac9ca684-7062-49c0-bbaf-5dd463e3ecfb/serve-healthcheck-canary/0.log" Jul 02 22:05:20.910184 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.910133 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:20.952188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:20.952147 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:20.952686 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:20.952408 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:21.168543 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.168437 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:21.226503 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.226474 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:21.304893 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.304844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:05:21.305064 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.304975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:05:21.307603 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.307577 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfx5z\" (UniqueName: \"kubernetes.io/projected/d8ea5660-2949-4e35-a277-45282a22550f-kube-api-access-kfx5z\") pod \"network-check-target-n7k68\" (UID: \"d8ea5660-2949-4e35-a277-45282a22550f\") " pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:05:21.307922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.307907 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 22:05:21.315280 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.315251 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 22:05:21.315431 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.315352 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs podName:4a4ca393-cf0e-4866-b73a-76ad7790a42d nodeName:}" failed. No retries permitted until 2026-07-02 22:06:25.31533033 +0000 UTC m=+130.299542399 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs") pod "network-metrics-daemon-sqlzp" (UID: "4a4ca393-cf0e-4866-b73a-76ad7790a42d") : secret "metrics-daemon-secret" not found Jul 02 22:05:21.406085 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.406047 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:05:21.406085 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.406090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:21.406368 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.406154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:21.406368 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.406261 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 22:05:21.406368 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.406275 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:21.406368 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.406335 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:25.406320101 +0000 UTC m=+70.390532172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : configmap "kube-rbac-proxy" not found Jul 02 22:05:21.406368 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:21.406365 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:25.406342179 +0000 UTC m=+70.390554244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:21.409099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.409082 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 22:05:21.416460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.416433 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lnnqk\"" Jul 02 22:05:21.418784 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.418725 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c995aee-a8c5-4d10-b49e-47da30a93d9b-original-pull-secret\") pod \"global-pull-secret-syncer-q55pk\" (UID: \"0c995aee-a8c5-4d10-b49e-47da30a93d9b\") " pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:05:21.424623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.424596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:05:21.554467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.554428 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7k68"] Jul 02 22:05:21.557587 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:21.557552 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ea5660_2949_4e35_a277_45282a22550f.slice/crio-a3c397060ebcbffcc0fdb9a8e2a569addde6d22c3a25e40f5bc30ea713903313 WatchSource:0}: Error finding container a3c397060ebcbffcc0fdb9a8e2a569addde6d22c3a25e40f5bc30ea713903313: Status 404 returned error can't find the container with id a3c397060ebcbffcc0fdb9a8e2a569addde6d22c3a25e40f5bc30ea713903313 Jul 02 22:05:21.607165 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.607133 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q55pk" Jul 02 22:05:21.737645 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.737609 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q55pk"] Jul 02 22:05:21.740592 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:21.740562 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c995aee_a8c5_4d10_b49e_47da30a93d9b.slice/crio-bb7883cd651c1c29278635f4a425471849a39d7ce3614bdb6d576f550cc77bdf WatchSource:0}: Error finding container bb7883cd651c1c29278635f4a425471849a39d7ce3614bdb6d576f550cc77bdf: Status 404 returned error can't find the container with id bb7883cd651c1c29278635f4a425471849a39d7ce3614bdb6d576f550cc77bdf Jul 02 22:05:21.923957 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.923920 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:21.956719 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.956624 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7k68" event={"ID":"d8ea5660-2949-4e35-a277-45282a22550f","Type":"ContainerStarted","Data":"24a81d18733c1bd8d8f8d758b774a457f878e1b911bdb26f026763cbb3dd7bd8"} Jul 02 22:05:21.956719 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.956681 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:05:21.956719 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.956699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7k68" event={"ID":"d8ea5660-2949-4e35-a277-45282a22550f","Type":"ContainerStarted","Data":"a3c397060ebcbffcc0fdb9a8e2a569addde6d22c3a25e40f5bc30ea713903313"} Jul 02 22:05:21.957742 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.957721 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q55pk" event={"ID":"0c995aee-a8c5-4d10-b49e-47da30a93d9b","Type":"ContainerStarted","Data":"bb7883cd651c1c29278635f4a425471849a39d7ce3614bdb6d576f550cc77bdf"} Jul 02 22:05:21.973797 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:21.973743 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n7k68" podStartSLOduration=65.973728256 podStartE2EDuration="1m5.973728256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:21.973030144 +0000 UTC m=+66.957242225" watchObservedRunningTime="2026-07-02 22:05:21.973728256 +0000 UTC m=+66.957940336" Jul 02 22:05:22.031918 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:22.031884 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:22.719697 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:22.719660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:22.719900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:22.719731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:22.719900 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:22.719835 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 22:05:22.719900 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:22.719893 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls podName:20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b nodeName:}" failed. No retries permitted until 2026-07-02 22:05:54.719881003 +0000 UTC m=+99.704093075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls") pod "dns-default-tpg8k" (UID: "20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b") : secret "dns-default-metrics-tls" not found Jul 02 22:05:22.720077 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:22.719840 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 22:05:22.720077 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:22.719936 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b967b955b-cbb9k: secret "image-registry-tls" not found Jul 02 22:05:22.720077 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:22.719977 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls podName:d4c3bd6c-f718-468e-965a-a0dcf457c8df nodeName:}" failed. No retries permitted until 2026-07-02 22:05:54.719965815 +0000 UTC m=+99.704177889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls") pod "image-registry-7b967b955b-cbb9k" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df") : secret "image-registry-tls" not found Jul 02 22:05:22.888003 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:22.887965 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:25.333530 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:25.333485 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:25.397554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:25.397518 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:25.443742 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:25.443707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:25.443941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:25.443818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:25.443941 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:25.443879 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:25.443941 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:25.443913 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 22:05:25.444068 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:25.443974 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:33.443951291 +0000 UTC m=+78.428163364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:25.444068 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:25.443997 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:33.443987525 +0000 UTC m=+78.428199590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : configmap "kube-rbac-proxy" not found Jul 02 22:05:26.223460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.223422 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:26.251731 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.251683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:26.251934 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.251858 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 22:05:26.251994 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.251943 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert podName:518c9c14-f70e-45a0-a566-80b717650204 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:42.251921255 +0000 UTC m=+87.236133339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-pk6n9" (UID: "518c9c14-f70e-45a0-a566-80b717650204") : secret "networking-console-plugin-cert" not found Jul 02 22:05:26.353100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.353064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:26.353100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.353105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.353147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.353170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353235 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353323 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353334 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:42.353290459 +0000 UTC m=+87.337502532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : secret "router-metrics-certs-default" not found Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353353 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle podName:bd81f4e8-4237-4f44-a4ea-8886d1bf8441 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:42.353343704 +0000 UTC m=+87.337555774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle") pod "router-default-7d48d4fd89-qjr8q" (UID: "bd81f4e8-4237-4f44-a4ea-8886d1bf8441") : configmap references non-existent config key: service-ca.crt Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353389 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls podName:3b02afa9-2fa0-45a2-9916-aefc2f901b8d nodeName:}" failed. No retries permitted until 2026-07-02 22:05:42.353375868 +0000 UTC m=+87.337587930 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kx49" (UID: "3b02afa9-2fa0-45a2-9916-aefc2f901b8d") : secret "samples-operator-tls" not found Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353323 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:26.353660 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.353495 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls podName:855a99f7-2255-47c6-831b-8b9eebbfc19c nodeName:}" failed. No retries permitted until 2026-07-02 22:05:42.353471619 +0000 UTC m=+87.337683679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c7crx" (UID: "855a99f7-2255-47c6-831b-8b9eebbfc19c") : secret "cluster-monitoring-operator-tls" not found Jul 02 22:05:26.927461 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.927372 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:26.927775 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.927761 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:26.927928 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:26.927909 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:26.977418 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.977382 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q55pk" event={"ID":"0c995aee-a8c5-4d10-b49e-47da30a93d9b","Type":"ContainerStarted","Data":"ac94e81794123414840469100564edd544561a825d0c69a78b31efe75755dab8"} Jul 02 22:05:26.993948 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:26.993881 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q55pk" podStartSLOduration=66.193510174 podStartE2EDuration="1m10.993862702s" totalImagesPullingTime="4.800352528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:26.992857308 +0000 UTC m=+71.977069392" watchObservedRunningTime="2026-07-02 22:05:26.993862702 +0000 UTC m=+71.978074783" Jul 02 22:05:27.969632 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:27.969598 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:28.081068 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:28.081030 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:28.181989 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:28.181951 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:28.283731 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:28.283635 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:29.258807 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:29.258770 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:29.651100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:29.651017 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:29.846615 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:29.846583 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:30.050484 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:30.050440 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:30.256250 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:30.256212 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:30.801732 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:30.801702 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:31.262542 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:31.262513 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:31.446704 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:31.446668 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:31.651567 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:31.651480 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:31.825688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:31.825653 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:31.885437 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:31.885405 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:32.054152 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:32.054117 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:32.483178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:32.483142 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:32.583873 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:32.583841 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:33.085604 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.085571 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:33.452239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.452200 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:33.519997 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.519962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:33.520143 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.520051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:33.520143 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:33.520117 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 22:05:33.520227 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:33.520197 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls podName:0344693d-b473-4240-a043-8a5e609a6793 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:49.520181149 +0000 UTC m=+94.504393207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5vzl" (UID: "0344693d-b473-4240-a043-8a5e609a6793") : secret "insights-runtime-extractor-tls" not found Jul 02 22:05:33.520586 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.520568 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0344693d-b473-4240-a043-8a5e609a6793-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:33.660219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.660183 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:33.729992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:33.729901 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:34.052264 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:34.052158 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:34.263902 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:34.263865 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:34.652553 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:34.652516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:34.709103 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:34.709065 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:35.254352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:35.254283 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:35.456090 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:35.456059 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:35.654738 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:35.654645 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:36.049819 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:36.049772 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:36.653194 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:36.653156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:37.063042 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:37.063010 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:37.645107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:37.645072 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:37.872178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:37.872145 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:38.251486 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:38.251455 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:38.447876 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:38.447839 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:38.678605 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:38.678520 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:40.595817 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:40.595786 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:41.022160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.022135 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:05:41.022584 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.022566 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/1.log" Jul 02 22:05:41.022685 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.022610 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" containerID="5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab" exitCode=255 Jul 02 22:05:41.022685 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.022676 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerDied","Data":"5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab"} Jul 02 22:05:41.022786 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.022738 2583 scope.go:117] "RemoveContainer" containerID="151fb2e74fddc846e24830d4d1d439013c631f29fba2aac28ab78868ba11d050" Jul 02 22:05:41.023076 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:41.023047 2583 scope.go:117] "RemoveContainer" containerID="5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab" Jul 02 22:05:41.023244 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:41.023225 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:42.027890 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.027860 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:05:42.297052 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.296953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:42.299351 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.299332 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/518c9c14-f70e-45a0-a566-80b717650204-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pk6n9\" (UID: \"518c9c14-f70e-45a0-a566-80b717650204\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:42.397986 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.397948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:42.398133 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.398009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:42.398173 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.398139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:42.398173 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.398166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:42.398735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.398704 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-service-ca-bundle\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:42.400565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.400530 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd81f4e8-4237-4f44-a4ea-8886d1bf8441-metrics-certs\") pod \"router-default-7d48d4fd89-qjr8q\" (UID: \"bd81f4e8-4237-4f44-a4ea-8886d1bf8441\") " pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:42.400565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.400537 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/855a99f7-2255-47c6-831b-8b9eebbfc19c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c7crx\" (UID: \"855a99f7-2255-47c6-831b-8b9eebbfc19c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:42.400732 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.400653 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b02afa9-2fa0-45a2-9916-aefc2f901b8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kx49\" (UID: \"3b02afa9-2fa0-45a2-9916-aefc2f901b8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:42.581536 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.581454 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-spfkp\"" Jul 02 22:05:42.589742 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.589719 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" Jul 02 22:05:42.601700 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.601673 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5sv64\"" Jul 02 22:05:42.608622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.608590 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n4fqx\"" Jul 02 22:05:42.609087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.609071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" Jul 02 22:05:42.617352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.617294 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:42.705750 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.705685 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g2wqj\"" Jul 02 22:05:42.729367 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.713541 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" Jul 02 22:05:42.782848 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.782817 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9"] Jul 02 22:05:42.788045 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:42.788011 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518c9c14_f70e_45a0_a566_80b717650204.slice/crio-1f5a174ca714c27b39bc6be6b9181addebfe3aa69cdceaf18b707230eb4531b1 WatchSource:0}: Error finding container 1f5a174ca714c27b39bc6be6b9181addebfe3aa69cdceaf18b707230eb4531b1: Status 404 returned error can't find the container with id 1f5a174ca714c27b39bc6be6b9181addebfe3aa69cdceaf18b707230eb4531b1 Jul 02 22:05:42.806037 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.806007 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49"] Jul 02 22:05:42.817168 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.817138 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d48d4fd89-qjr8q"] Jul 02 22:05:42.820670 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:42.820638 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd81f4e8_4237_4f44_a4ea_8886d1bf8441.slice/crio-136f490b668bec73beb45c3f16a990431e881d9ece7d1dabd4b3d3db59887e63 WatchSource:0}: Error finding container 136f490b668bec73beb45c3f16a990431e881d9ece7d1dabd4b3d3db59887e63: Status 404 returned error can't find the container with id 136f490b668bec73beb45c3f16a990431e881d9ece7d1dabd4b3d3db59887e63 Jul 02 22:05:42.890978 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:42.890822 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx"] Jul 02 22:05:42.894070 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:42.894039 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855a99f7_2255_47c6_831b_8b9eebbfc19c.slice/crio-e83dfd27a39e23e0fc89dff716d2dcf0c84e69c39a9d3ba3da53318b2d195fdd WatchSource:0}: Error finding container e83dfd27a39e23e0fc89dff716d2dcf0c84e69c39a9d3ba3da53318b2d195fdd: Status 404 returned error can't find the container with id e83dfd27a39e23e0fc89dff716d2dcf0c84e69c39a9d3ba3da53318b2d195fdd Jul 02 22:05:43.035155 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.035116 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" event={"ID":"3b02afa9-2fa0-45a2-9916-aefc2f901b8d","Type":"ContainerStarted","Data":"f0c6049ef85877c48aa5af0e1abd3296791c35edf4d9fa2227ec619f6a87e7b4"} Jul 02 22:05:43.036417 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.036393 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" event={"ID":"855a99f7-2255-47c6-831b-8b9eebbfc19c","Type":"ContainerStarted","Data":"e83dfd27a39e23e0fc89dff716d2dcf0c84e69c39a9d3ba3da53318b2d195fdd"} Jul 02 22:05:43.037982 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.037953 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" event={"ID":"bd81f4e8-4237-4f44-a4ea-8886d1bf8441","Type":"ContainerStarted","Data":"30cc1e21ae32407c4a31372d57ac5969673d16458366b16ad37a38d5b0e425e9"} Jul 02 22:05:43.038083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.038006 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" event={"ID":"bd81f4e8-4237-4f44-a4ea-8886d1bf8441","Type":"ContainerStarted","Data":"136f490b668bec73beb45c3f16a990431e881d9ece7d1dabd4b3d3db59887e63"} Jul 02 22:05:43.039115 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.039094 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" event={"ID":"518c9c14-f70e-45a0-a566-80b717650204","Type":"ContainerStarted","Data":"1f5a174ca714c27b39bc6be6b9181addebfe3aa69cdceaf18b707230eb4531b1"} Jul 02 22:05:43.095000 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.094895 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" podStartSLOduration=33.094875526 podStartE2EDuration="33.094875526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:43.089962721 +0000 UTC m=+88.074174803" watchObservedRunningTime="2026-07-02 22:05:43.094875526 +0000 UTC m=+88.079087607" Jul 02 22:05:43.163917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.163878 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:43.267641 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.267604 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:43.617467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.617427 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:43.619935 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:43.619911 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:44.043475 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:44.043448 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:44.045060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:44.044855 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d48d4fd89-qjr8q" Jul 02 22:05:45.309817 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:45.309773 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:45.541210 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:45.541181 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:45.865270 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:45.865178 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:45.966702 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:45.966670 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:46.043602 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.043569 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh"] Jul 02 22:05:46.046846 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.046827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:46.054528 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.054498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" event={"ID":"3b02afa9-2fa0-45a2-9916-aefc2f901b8d","Type":"ContainerStarted","Data":"9f98b198073ba7310d81b7efd6578473909f379ca198ea8b4f7a1cc2c3a39422"} Jul 02 22:05:46.054660 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.054556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" event={"ID":"3b02afa9-2fa0-45a2-9916-aefc2f901b8d","Type":"ContainerStarted","Data":"d6f75bef1c5845ed8632bf20493701f1effbb3f7bb4be8a093070cf9a1bf52cf"} Jul 02 22:05:46.056219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.056196 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" event={"ID":"855a99f7-2255-47c6-831b-8b9eebbfc19c","Type":"ContainerStarted","Data":"711a0b363cbcc3abf1b34dd98c3e533ac892136ed84d670ef4de209b338cdb3c"} Jul 02 22:05:46.057777 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.057759 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" event={"ID":"518c9c14-f70e-45a0-a566-80b717650204","Type":"ContainerStarted","Data":"f8ad67ac44c881248b46f61870163932df5dee4570120d169f0bed63d190d724"} Jul 02 22:05:46.068100 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:46.068071 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-operator-admission-webhook-d559d66d5-85nrh\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:46.068231 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:46.068127 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-tls\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret" Jul 02 22:05:46.068231 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:46.068202 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-dockercfg-bxkbb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bxkbb\"" type="*v1.Secret" Jul 02 22:05:46.081311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.081262 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:46.092554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.092531 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh"] Jul 02 22:05:46.112515 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.112466 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c7crx" podStartSLOduration=33.515337942 podStartE2EDuration="36.112450876s" totalImagesPullingTime="2.597112934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:46.110231772 +0000 UTC m=+91.094443853" watchObservedRunningTime="2026-07-02 22:05:46.112450876 +0000 UTC m=+91.096663008" Jul 02 22:05:46.162183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.162068 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kx49" podStartSLOduration=33.52841045 podStartE2EDuration="36.162050729s" totalImagesPullingTime="2.633640279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:46.161422682 +0000 UTC m=+91.145634764" watchObservedRunningTime="2026-07-02 22:05:46.162050729 +0000 UTC m=+91.146262810" Jul 02 22:05:46.180197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.180142 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pk6n9" podStartSLOduration=33.477023868 podStartE2EDuration="36.180125506s" totalImagesPullingTime="2.703101638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:46.178818773 +0000 UTC m=+91.163030854" watchObservedRunningTime="2026-07-02 22:05:46.180125506 +0000 UTC m=+91.164337587" Jul 02 22:05:46.230977 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.230934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-85nrh\" (UID: \"d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:46.264466 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.264434 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:46.331863 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.331822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-85nrh\" (UID: \"d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:46.365742 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.365704 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:46.430925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.430837 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:46.916668 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.916635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 22:05:46.924729 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.924693 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-85nrh\" (UID: \"d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:46.927439 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.927420 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:46.927781 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:46.927769 2583 scope.go:117] "RemoveContainer" containerID="5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab" Jul 02 22:05:46.927933 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:46.927914 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:47.553127 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:47.553093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bxkbb\"" Jul 02 22:05:47.556312 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:47.556286 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:47.717179 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:47.717146 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh"] Jul 02 22:05:47.720076 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:47.720042 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a7c966_eaa4_4fb6_a3fa_ba4ff46e13e4.slice/crio-39a3ca2a7c2e47c59a74bfeabdec33d2f230123a97031253364216a01383ddc0 WatchSource:0}: Error finding container 39a3ca2a7c2e47c59a74bfeabdec33d2f230123a97031253364216a01383ddc0: Status 404 returned error can't find the container with id 39a3ca2a7c2e47c59a74bfeabdec33d2f230123a97031253364216a01383ddc0 Jul 02 22:05:48.066674 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:48.066638 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" event={"ID":"d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4","Type":"ContainerStarted","Data":"39a3ca2a7c2e47c59a74bfeabdec33d2f230123a97031253364216a01383ddc0"} Jul 02 22:05:49.556034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:49.556003 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:49.558287 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:49.558268 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0344693d-b473-4240-a043-8a5e609a6793-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5vzl\" (UID: \"0344693d-b473-4240-a043-8a5e609a6793\") " pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:49.752948 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:49.752910 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5vzl" Jul 02 22:05:49.903286 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:49.903253 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5vzl"] Jul 02 22:05:49.906853 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:49.906820 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0344693d_b473_4240_a043_8a5e609a6793.slice/crio-86a7d0ca40cc36ce4ba0bf4854843bb89ebeb4a86a1d8df64eb67ee01e359a73 WatchSource:0}: Error finding container 86a7d0ca40cc36ce4ba0bf4854843bb89ebeb4a86a1d8df64eb67ee01e359a73: Status 404 returned error can't find the container with id 86a7d0ca40cc36ce4ba0bf4854843bb89ebeb4a86a1d8df64eb67ee01e359a73 Jul 02 22:05:50.075091 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.074992 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" event={"ID":"d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4","Type":"ContainerStarted","Data":"7999de6c2217e22783058ade8e15f4329eec02a84756217b2d54d4caddd167a7"} Jul 02 22:05:50.075091 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.075060 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:50.076181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.076156 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5vzl" event={"ID":"0344693d-b473-4240-a043-8a5e609a6793","Type":"ContainerStarted","Data":"86a7d0ca40cc36ce4ba0bf4854843bb89ebeb4a86a1d8df64eb67ee01e359a73"} Jul 02 22:05:50.079829 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.079812 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" Jul 02 22:05:50.091327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.091263 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-85nrh" podStartSLOduration=2.2660563959999998 podStartE2EDuration="4.091247737s" totalImagesPullingTime="1.825191341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:50.089900372 +0000 UTC m=+95.074112454" watchObservedRunningTime="2026-07-02 22:05:50.091247737 +0000 UTC m=+95.075459818" Jul 02 22:05:50.909428 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.909389 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:05:50.909917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:50.909899 2583 scope.go:117] "RemoveContainer" containerID="5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab" Jul 02 22:05:50.910106 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:50.910084 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-gd55z_openshift-console-operator(3ae65cac-b4f8-4b54-944b-b6fd61f251c0)\"" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" podUID="3ae65cac-b4f8-4b54-944b-b6fd61f251c0" Jul 02 22:05:51.087110 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:51.087076 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:51.164070 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:51.163923 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:51.233214 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:51.233180 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:52.422786 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:52.422697 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:52.492173 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:52.492136 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:52.964210 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:52.964182 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n7k68" Jul 02 22:05:54.802932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:54.802899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:54.803266 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:54.802941 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:54.805237 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:54.805212 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b-metrics-tls\") pod \"dns-default-tpg8k\" (UID: \"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b\") " pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:54.805468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:54.805450 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"image-registry-7b967b955b-cbb9k\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:54.825019 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:54.824939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:55.047625 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.047593 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qk2ld\"" Jul 02 22:05:55.054854 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.054833 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:55.057521 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.057501 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8wxkj\"" Jul 02 22:05:55.065834 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.065756 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:55.101622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.101590 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5vzl" event={"ID":"0344693d-b473-4240-a043-8a5e609a6793","Type":"ContainerStarted","Data":"dbdf2c374db9230cde79661bd33b6c89d261d98faa1bec959c09fbe0ac0a4a65"} Jul 02 22:05:55.206559 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.206524 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:05:55.210204 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:55.210179 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c3bd6c_f718_468e_965a_a0dcf457c8df.slice/crio-9d2d61ce2fd7e7bca770303c504460b54bd3afa2015ea8f4c28dcfb91e75cbb9 WatchSource:0}: Error finding container 9d2d61ce2fd7e7bca770303c504460b54bd3afa2015ea8f4c28dcfb91e75cbb9: Status 404 returned error can't find the container with id 9d2d61ce2fd7e7bca770303c504460b54bd3afa2015ea8f4c28dcfb91e75cbb9 Jul 02 22:05:55.218094 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:55.218052 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tpg8k"] Jul 02 22:05:55.222205 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:55.222171 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ef0b8f_6617_4fd3_b0b4_82c2a5ca363b.slice/crio-0f4a83667a01e55d8c8d0d75f73abd0b45ce894e94a317b911f515da8dd3fb53 WatchSource:0}: Error finding container 0f4a83667a01e55d8c8d0d75f73abd0b45ce894e94a317b911f515da8dd3fb53: Status 404 returned error can't find the container with id 0f4a83667a01e55d8c8d0d75f73abd0b45ce894e94a317b911f515da8dd3fb53 Jul 02 22:05:56.108483 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.108439 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5vzl" event={"ID":"0344693d-b473-4240-a043-8a5e609a6793","Type":"ContainerStarted","Data":"ba573369cab21f1829ab246bc8c6dfd406be52e9123b85eadc435a21a0f70368"} Jul 02 22:05:56.110360 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.110319 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpg8k" event={"ID":"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b","Type":"ContainerStarted","Data":"0f4a83667a01e55d8c8d0d75f73abd0b45ce894e94a317b911f515da8dd3fb53"} Jul 02 22:05:56.112315 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.112264 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" event={"ID":"d4c3bd6c-f718-468e-965a-a0dcf457c8df","Type":"ContainerStarted","Data":"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218"} Jul 02 22:05:56.112410 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.112341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" event={"ID":"d4c3bd6c-f718-468e-965a-a0dcf457c8df","Type":"ContainerStarted","Data":"9d2d61ce2fd7e7bca770303c504460b54bd3afa2015ea8f4c28dcfb91e75cbb9"} Jul 02 22:05:56.112516 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.112500 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:05:56.134746 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.134683 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" podStartSLOduration=100.134662851 podStartE2EDuration="1m40.134662851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:56.133236156 +0000 UTC m=+101.117448238" watchObservedRunningTime="2026-07-02 22:05:56.134662851 +0000 UTC m=+101.118874932" Jul 02 22:05:56.231241 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.231084 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.462394 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.462354 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.522064 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.522032 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2pmzx"] Jul 02 22:05:56.526064 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.526039 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.529122 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.529087 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 22:05:56.529459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.529258 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 22:05:56.529459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.529417 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 22:05:56.529459 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.529450 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 22:05:56.530463 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.530441 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-np5tg\"" Jul 02 22:05:56.560596 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.560555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.620019 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.619977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-wtmp\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-sys\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-textfile\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-root\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-tls\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cszsr\" (UniqueName: \"kubernetes.io/projected/11a10297-7920-4ee0-9159-74cbcf53b708-kube-api-access-cszsr\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-metrics-client-ca\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.620467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.620424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.623952 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.623904 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.721800 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.721701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-metrics-client-ca\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.721800 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.721744 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.721800 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.721791 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-wtmp\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722057 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.721813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722057 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.721980 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-wtmp\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722057 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-sys\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-textfile\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-root\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-tls\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722212 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722191 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cszsr\" (UniqueName: \"kubernetes.io/projected/11a10297-7920-4ee0-9159-74cbcf53b708-kube-api-access-cszsr\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722458 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722535 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-root\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a10297-7920-4ee0-9159-74cbcf53b708-metrics-client-ca\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a10297-7920-4ee0-9159-74cbcf53b708-sys\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.722637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.722578 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-textfile\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.724931 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.724906 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.725294 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.725268 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a10297-7920-4ee0-9159-74cbcf53b708-node-exporter-tls\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.726055 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.725950 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.731725 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.731671 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cszsr\" (UniqueName: \"kubernetes.io/projected/11a10297-7920-4ee0-9159-74cbcf53b708-kube-api-access-cszsr\") pod \"node-exporter-2pmzx\" (UID: \"11a10297-7920-4ee0-9159-74cbcf53b708\") " pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:56.824825 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.824786 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:56.838136 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:56.838104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2pmzx" Jul 02 22:05:57.004869 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.004792 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:57.328485 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.328407 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:57.457371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.457277 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:57.553497 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.553455 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:57.564778 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.564748 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:05:57.688434 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.688405 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.691670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691639 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 22:05:57.691670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691653 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 22:05:57.691901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691683 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fprrc\"" Jul 02 22:05:57.691901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691644 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 22:05:57.691901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691640 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 22:05:57.691901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.691655 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 22:05:57.692099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.692011 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 22:05:57.692099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.692026 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 22:05:57.692099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.692042 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 22:05:57.692373 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.692358 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 22:05:57.693870 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.693852 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:05:57.730774 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730742 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.730774 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730775 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.730983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.730983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730819 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xzth\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.730983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.730983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.730979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731058 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731122 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.731348 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.731140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831569 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831794 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.831874 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:57.831910 2583 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831920 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.831945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xzth\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:57.831983 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls podName:b8b6928a-bb3b-4702-8944-c1224c2eeb26 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:58.331961115 +0000 UTC m=+103.316173179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26") : secret "alertmanager-main-tls" not found Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.832231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.832319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.832365 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.832397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.833776 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:05:57.833905 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle podName:b8b6928a-bb3b-4702-8944-c1224c2eeb26 nodeName:}" failed. No retries permitted until 2026-07-02 22:05:58.333886306 +0000 UTC m=+103.318098367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26") : configmap references non-existent config key: ca-bundle.crt Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.834615 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.834796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.834922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.834836 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.835770 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.835748 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.837030 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.836606 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.837030 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.836677 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.837030 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.836987 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.837878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.837804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.838781 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.838738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.840827 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.840808 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xzth\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:57.999253 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:57.999159 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:58.100727 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.100688 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:58.337571 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.337487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:58.337571 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.337549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:58.338205 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.338181 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:58.340018 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.339999 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:58.546786 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:58.546731 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a10297_7920_4ee0_9159_74cbcf53b708.slice/crio-4c5bacd8d39a0422759611b4f161a3c430a89cf17dedf8813a178eaf26e2a3ea WatchSource:0}: Error finding container 4c5bacd8d39a0422759611b4f161a3c430a89cf17dedf8813a178eaf26e2a3ea: Status 404 returned error can't find the container with id 4c5bacd8d39a0422759611b4f161a3c430a89cf17dedf8813a178eaf26e2a3ea Jul 02 22:05:58.599320 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.599274 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:05:58.762789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:58.762762 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:05:58.766496 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:05:58.766469 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b6928a_bb3b_4702_8944_c1224c2eeb26.slice/crio-ceb28ece64115d9c9a089664d81196df6e357cd23cfc9487f246cb5c87ea3f0f WatchSource:0}: Error finding container ceb28ece64115d9c9a089664d81196df6e357cd23cfc9487f246cb5c87ea3f0f: Status 404 returned error can't find the container with id ceb28ece64115d9c9a089664d81196df6e357cd23cfc9487f246cb5c87ea3f0f Jul 02 22:05:59.126510 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.126471 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pmzx" event={"ID":"11a10297-7920-4ee0-9159-74cbcf53b708","Type":"ContainerStarted","Data":"4c5bacd8d39a0422759611b4f161a3c430a89cf17dedf8813a178eaf26e2a3ea"} Jul 02 22:05:59.127948 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.127919 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"ceb28ece64115d9c9a089664d81196df6e357cd23cfc9487f246cb5c87ea3f0f"} Jul 02 22:05:59.130462 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.130430 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5vzl" event={"ID":"0344693d-b473-4240-a043-8a5e609a6793","Type":"ContainerStarted","Data":"a12251ca134b61be7392639af48be7061626f1e03cbcbe498acfe3c2147f2bb0"} Jul 02 22:05:59.132415 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.132392 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpg8k" event={"ID":"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b","Type":"ContainerStarted","Data":"88c3859a4daf354a7eeb6c38f3b591999d046090b3d167ee2b52e37d3727d34e"} Jul 02 22:05:59.132513 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.132438 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpg8k" event={"ID":"20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b","Type":"ContainerStarted","Data":"f4ec57d754f62e8f8c398451e9731905231347922e66ff6c8e14e306f46df28b"} Jul 02 22:05:59.132586 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.132572 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tpg8k" Jul 02 22:05:59.155337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.155260 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z5vzl" podStartSLOduration=33.651861384 podStartE2EDuration="42.155237753s" totalImagesPullingTime="8.503376369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:05:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:59.153423179 +0000 UTC m=+104.137635262" watchObservedRunningTime="2026-07-02 22:05:59.155237753 +0000 UTC m=+104.139449838" Jul 02 22:05:59.194575 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.194509 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tpg8k" podStartSLOduration=65.868800004 podStartE2EDuration="1m9.194491674s" totalImagesPullingTime="3.32569167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:05:59.194178979 +0000 UTC m=+104.178391061" watchObservedRunningTime="2026-07-02 22:05:59.194491674 +0000 UTC m=+104.178703746" Jul 02 22:05:59.319612 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.319572 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:59.424257 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.424151 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:59.483818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.483784 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:59.618601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.618566 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:59.958839 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:05:59.958816 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:00.139716 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.139618 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc" exitCode=0 Jul 02 22:06:00.139716 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.139702 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc"} Jul 02 22:06:00.141408 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.141335 2583 generic.go:356] "Generic (PLEG): container finished" podID="11a10297-7920-4ee0-9159-74cbcf53b708" containerID="c7e466df112e58e79593c2b42bbd7a98d4eab2312e1fd8b245358b7cd2a97dec" exitCode=0 Jul 02 22:06:00.141468 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.141419 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pmzx" event={"ID":"11a10297-7920-4ee0-9159-74cbcf53b708","Type":"ContainerDied","Data":"c7e466df112e58e79593c2b42bbd7a98d4eab2312e1fd8b245358b7cd2a97dec"} Jul 02 22:06:00.798197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.798163 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76958644b7-8rnf7"] Jul 02 22:06:00.803063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.803033 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.806696 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.806674 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-thrkv\"" Jul 02 22:06:00.806830 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.806759 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 22:06:00.806970 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.806950 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 22:06:00.808341 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.808260 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 22:06:00.808801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.808783 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 22:06:00.808997 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.808803 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f590b49m9ie5k\"" Jul 02 22:06:00.814688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.814654 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76958644b7-8rnf7"] Jul 02 22:06:00.864728 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-client-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.864893 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-tls\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.864893 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b24ebc97-8c8e-486a-83f7-83b177197cd7-audit-log\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.864893 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864858 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2cmx\" (UniqueName: \"kubernetes.io/projected/b24ebc97-8c8e-486a-83f7-83b177197cd7-kube-api-access-v2cmx\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.865016 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864893 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.865016 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864948 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-metrics-server-audit-profiles\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.865016 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.864981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-client-certs\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966008 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.965969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-metrics-server-audit-profiles\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966008 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-client-certs\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-client-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-tls\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966165 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b24ebc97-8c8e-486a-83f7-83b177197cd7-audit-log\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966185 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2cmx\" (UniqueName: \"kubernetes.io/projected/b24ebc97-8c8e-486a-83f7-83b177197cd7-kube-api-access-v2cmx\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966651 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b24ebc97-8c8e-486a-83f7-83b177197cd7-audit-log\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.966956 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.966919 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.967128 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.967103 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b24ebc97-8c8e-486a-83f7-83b177197cd7-metrics-server-audit-profiles\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.969039 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.969018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-tls\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.969146 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.969125 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-client-ca-bundle\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.969271 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.969248 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b24ebc97-8c8e-486a-83f7-83b177197cd7-secret-metrics-server-client-certs\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:00.976655 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:00.976603 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2cmx\" (UniqueName: \"kubernetes.io/projected/b24ebc97-8c8e-486a-83f7-83b177197cd7-kube-api-access-v2cmx\") pod \"metrics-server-76958644b7-8rnf7\" (UID: \"b24ebc97-8c8e-486a-83f7-83b177197cd7\") " pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:01.116872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.116763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:01.148700 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.148656 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pmzx" event={"ID":"11a10297-7920-4ee0-9159-74cbcf53b708","Type":"ContainerStarted","Data":"9d45c10e2ba1d482a5399f46af0b650a80727bd9dd1b1a6bf3de9f1c1617beaa"} Jul 02 22:06:01.148864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.148737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pmzx" event={"ID":"11a10297-7920-4ee0-9159-74cbcf53b708","Type":"ContainerStarted","Data":"1405393c283cd7a87642a42d29a07e82d68a23e24fa49b51d70845e1297c8626"} Jul 02 22:06:01.176393 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.176343 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2pmzx" podStartSLOduration=4.167922897 podStartE2EDuration="5.176323579s" totalImagesPullingTime="937.357895ms" totalInitContainerRuntime="71.042787ms" isStatefulPod=true podCreationTimestamp="2026-07-02 22:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:01.175632385 +0000 UTC m=+106.159844466" watchObservedRunningTime="2026-07-02 22:06:01.176323579 +0000 UTC m=+106.160535651" Jul 02 22:06:01.268674 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.268644 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76958644b7-8rnf7"] Jul 02 22:06:01.271697 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:06:01.271667 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24ebc97_8c8e_486a_83f7_83b177197cd7.slice/crio-ccfeef77d3c4583ad86c92beda36fd6693c0bd5294a2fe0dbf08616dca9f0627 WatchSource:0}: Error finding container ccfeef77d3c4583ad86c92beda36fd6693c0bd5294a2fe0dbf08616dca9f0627: Status 404 returned error can't find the container with id ccfeef77d3c4583ad86c92beda36fd6693c0bd5294a2fe0dbf08616dca9f0627 Jul 02 22:06:01.275739 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.275714 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:01.539761 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.539724 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:01.595524 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:01.595492 2583 scope.go:117] "RemoveContainer" containerID="5742e33cff973103d774f284e0adf5df41c6a5b7687c7fe22006c8ea4dac30ab" Jul 02 22:06:02.161975 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.161874 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:06:02.162628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.162009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" event={"ID":"3ae65cac-b4f8-4b54-944b-b6fd61f251c0","Type":"ContainerStarted","Data":"9220ab281a8a554210950e3be87b9217f94d702e96c205196e0f2f39918d951d"} Jul 02 22:06:02.163087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.162981 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:06:02.168416 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.168294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139"} Jul 02 22:06:02.169146 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.169101 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-gd55z" Jul 02 22:06:02.170811 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.170783 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" event={"ID":"b24ebc97-8c8e-486a-83f7-83b177197cd7","Type":"ContainerStarted","Data":"ccfeef77d3c4583ad86c92beda36fd6693c0bd5294a2fe0dbf08616dca9f0627"} Jul 02 22:06:02.310544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.310518 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-9m6tc"] Jul 02 22:06:02.314412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.314214 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:02.317595 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.317571 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 22:06:02.317723 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.317652 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 22:06:02.318220 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.317967 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sc49d\"" Jul 02 22:06:02.322073 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.321963 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:02.325388 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.325365 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-9m6tc"] Jul 02 22:06:02.379022 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.378965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8slg\" (UniqueName: \"kubernetes.io/projected/c3de5eeb-49f4-48d7-93b1-ab5140d5e407-kube-api-access-m8slg\") pod \"downloads-685bfb5f64-9m6tc\" (UID: \"c3de5eeb-49f4-48d7-93b1-ab5140d5e407\") " pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:02.416658 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.416615 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:02.480337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.480270 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8slg\" (UniqueName: \"kubernetes.io/projected/c3de5eeb-49f4-48d7-93b1-ab5140d5e407-kube-api-access-m8slg\") pod \"downloads-685bfb5f64-9m6tc\" (UID: \"c3de5eeb-49f4-48d7-93b1-ab5140d5e407\") " pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:02.493175 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.493141 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8slg\" (UniqueName: \"kubernetes.io/projected/c3de5eeb-49f4-48d7-93b1-ab5140d5e407-kube-api-access-m8slg\") pod \"downloads-685bfb5f64-9m6tc\" (UID: \"c3de5eeb-49f4-48d7-93b1-ab5140d5e407\") " pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:02.516080 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.515988 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:02.616693 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.616655 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:02.627560 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.627528 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:02.742245 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.742211 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:02.820772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.820747 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-9m6tc"] Jul 02 22:06:02.887104 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:02.887068 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:03.092359 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:06:03.092275 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3de5eeb_49f4_48d7_93b1_ab5140d5e407.slice/crio-c5f91a024e55e634f95b051732c14382b6689dc5677aeea590c5601c1beabc3f WatchSource:0}: Error finding container c5f91a024e55e634f95b051732c14382b6689dc5677aeea590c5601c1beabc3f: Status 404 returned error can't find the container with id c5f91a024e55e634f95b051732c14382b6689dc5677aeea590c5601c1beabc3f Jul 02 22:06:03.176103 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.176073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-9m6tc" event={"ID":"c3de5eeb-49f4-48d7-93b1-ab5140d5e407","Type":"ContainerStarted","Data":"c5f91a024e55e634f95b051732c14382b6689dc5677aeea590c5601c1beabc3f"} Jul 02 22:06:03.184835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.184802 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666"} Jul 02 22:06:03.184987 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.184907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3"} Jul 02 22:06:03.184987 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.184935 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a"} Jul 02 22:06:03.184987 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.184963 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897"} Jul 02 22:06:03.409064 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.408956 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:03.624477 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.624438 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:03.706536 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.706478 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:03.830829 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.830794 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:03.924777 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:03.924740 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:04.009375 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.009253 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:04.177981 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.177949 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:04.192577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.192545 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerStarted","Data":"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d"} Jul 02 22:06:04.194007 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.193977 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" event={"ID":"b24ebc97-8c8e-486a-83f7-83b177197cd7","Type":"ContainerStarted","Data":"9a2249ac6efc5c78d4866e177752392080a1244bdbdbf4ef0caf9828c5aa6554"} Jul 02 22:06:04.247739 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.247659 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.140038984 podStartE2EDuration="7.24763736s" totalImagesPullingTime="4.091787211s" totalInitContainerRuntime="15.811165ms" isStatefulPod=false podCreationTimestamp="2026-07-02 22:05:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:04.24587844 +0000 UTC m=+109.230090532" watchObservedRunningTime="2026-07-02 22:06:04.24763736 +0000 UTC m=+109.231849441" Jul 02 22:06:04.269176 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.269014 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" podStartSLOduration=2.400703224 podStartE2EDuration="4.268988737s" totalImagesPullingTime="1.868285513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:04.268728422 +0000 UTC m=+109.252940503" watchObservedRunningTime="2026-07-02 22:06:04.268988737 +0000 UTC m=+109.253200819" Jul 02 22:06:04.278710 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:04.278624 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:05.600279 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:05.600233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:05.662235 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:05.662197 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:05.978239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:05.978204 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:06:06.061980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:06.061944 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:06.161931 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:06.161894 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:06.224970 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:06.224940 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:07.415631 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.415592 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:07.419448 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.419423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.422527 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.422507 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 22:06:07.423923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.423701 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 22:06:07.423923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.423739 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 22:06:07.423923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.423701 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 22:06:07.423923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.423751 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 22:06:07.423923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.423850 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tmjvg\"" Jul 02 22:06:07.431952 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.431927 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:07.532925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.532887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.532925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.532930 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.533178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.532950 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.533178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.533021 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8ftq\" (UniqueName: \"kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.533178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.533119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.533178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.533154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.633880 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.633840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.633889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.633922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.633950 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ftq\" (UniqueName: \"kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.633981 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.634002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.634762 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634939 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.634776 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.634939 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.634896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.636952 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.636924 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.637060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.636947 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.644618 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.644587 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8ftq\" (UniqueName: \"kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq\") pod \"console-7d44b866bc-qkhtx\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.731492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.731452 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:07.894904 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:07.894873 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:07.897041 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:06:07.897014 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55ff76a_e01c_4945_9d36_bce93fd917a4.slice/crio-c116a4ccd71d638973f92bdc0b7826c44d000829151c6a691ba764f9ef766e2c WatchSource:0}: Error finding container c116a4ccd71d638973f92bdc0b7826c44d000829151c6a691ba764f9ef766e2c: Status 404 returned error can't find the container with id c116a4ccd71d638973f92bdc0b7826c44d000829151c6a691ba764f9ef766e2c Jul 02 22:06:08.213832 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:08.213790 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d44b866bc-qkhtx" event={"ID":"b55ff76a-e01c-4945-9d36-bce93fd917a4","Type":"ContainerStarted","Data":"c116a4ccd71d638973f92bdc0b7826c44d000829151c6a691ba764f9ef766e2c"} Jul 02 22:06:08.911559 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:08.911521 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:09.013833 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:09.013770 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:09.143952 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:09.143909 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tpg8k" Jul 02 22:06:09.279902 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:09.279869 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:09.378290 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:09.378252 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:12.238115 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:12.238072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d44b866bc-qkhtx" event={"ID":"b55ff76a-e01c-4945-9d36-bce93fd917a4","Type":"ContainerStarted","Data":"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b"} Jul 02 22:06:12.262843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:12.262789 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d44b866bc-qkhtx" podStartSLOduration=1.487293511 podStartE2EDuration="5.262773003s" totalImagesPullingTime="3.775479492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:12.26033266 +0000 UTC m=+117.244544743" watchObservedRunningTime="2026-07-02 22:06:12.262773003 +0000 UTC m=+117.246985140" Jul 02 22:06:14.145025 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:14.144975 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:14.245675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:14.245636 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:15.008243 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.008207 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:06:15.013240 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.013211 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.025009 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.024977 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:06:15.025275 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.025248 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 22:06:15.105448 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105465 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105498 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c85mx\" (UniqueName: \"kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105604 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105781 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.105781 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.105716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207130 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207130 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c85mx\" (UniqueName: \"kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207937 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.207937 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.207862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.208528 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.208499 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.213917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.208963 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.213917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.209048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.213917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.209804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.213917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.210949 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.215000 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.214972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.217126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.217101 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c85mx\" (UniqueName: \"kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx\") pod \"console-7766fb46b5-85lnf\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.327558 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.327467 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:15.481257 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.481226 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:06:15.484772 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:06:15.484744 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6beb7a4_c640_4301_acba_afc0396881ae.slice/crio-e6e4b2ab2293e940d61c28fc43f38857883700c3019c688f73b4888eb0289c14 WatchSource:0}: Error finding container e6e4b2ab2293e940d61c28fc43f38857883700c3019c688f73b4888eb0289c14: Status 404 returned error can't find the container with id e6e4b2ab2293e940d61c28fc43f38857883700c3019c688f73b4888eb0289c14 Jul 02 22:06:15.986228 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:15.986199 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:06:16.215983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:16.215948 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:16.258796 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:16.258713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766fb46b5-85lnf" event={"ID":"f6beb7a4-c640-4301-acba-afc0396881ae","Type":"ContainerStarted","Data":"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e"} Jul 02 22:06:16.258796 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:16.258765 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766fb46b5-85lnf" event={"ID":"f6beb7a4-c640-4301-acba-afc0396881ae","Type":"ContainerStarted","Data":"e6e4b2ab2293e940d61c28fc43f38857883700c3019c688f73b4888eb0289c14"} Jul 02 22:06:16.270972 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:16.270934 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:16.277675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:16.277626 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7766fb46b5-85lnf" podStartSLOduration=2.277609401 podStartE2EDuration="2.277609401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:06:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:16.276507033 +0000 UTC m=+121.260719114" watchObservedRunningTime="2026-07-02 22:06:16.277609401 +0000 UTC m=+121.261821482" Jul 02 22:06:17.732025 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:17.731986 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:17.732492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:17.732055 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:17.738859 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:17.738832 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:18.001026 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:18.000937 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:18.072353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:18.072269 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:18.273895 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:18.273823 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:18.703663 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:18.703621 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:18.806643 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:18.806577 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:19.136610 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:19.136521 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:19.254621 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:19.254584 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:21.117247 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:21.117207 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:21.117754 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:21.117292 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:23.295363 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:23.295249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-9m6tc" event={"ID":"c3de5eeb-49f4-48d7-93b1-ab5140d5e407","Type":"ContainerStarted","Data":"f918b122ab503dfa5743aa90be9869300be476eb6db67957c9224e60fe1baa8b"} Jul 02 22:06:23.295808 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:23.295478 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:23.309762 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:23.309731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-9m6tc" Jul 02 22:06:23.318721 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:23.318660 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-9m6tc" podStartSLOduration=1.4034247340000001 podStartE2EDuration="21.318641729s" totalImagesPullingTime="19.915216995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:23.317081889 +0000 UTC m=+128.301293990" watchObservedRunningTime="2026-07-02 22:06:23.318641729 +0000 UTC m=+128.302853809" Jul 02 22:06:24.217787 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:24.217747 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:24.278261 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:24.278220 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:25.328160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.328118 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:25.328658 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.328202 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:25.335000 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.334969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:25.413403 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.413364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:06:25.416227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.416195 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a4ca393-cf0e-4866-b73a-76ad7790a42d-metrics-certs\") pod \"network-metrics-daemon-sqlzp\" (UID: \"4a4ca393-cf0e-4866-b73a-76ad7790a42d\") " pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:06:25.609937 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.609858 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ddmbm\"" Jul 02 22:06:25.617065 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.617035 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqlzp" Jul 02 22:06:25.617239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.617096 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:25.681514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.681468 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:25.807859 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:25.807825 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqlzp"] Jul 02 22:06:25.810887 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:06:25.810848 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4ca393_cf0e_4866_b73a_76ad7790a42d.slice/crio-532a8f698ccce99cfcc28920b5ae4b138b48cffd35945f12da9f9bf91fb3a4b3 WatchSource:0}: Error finding container 532a8f698ccce99cfcc28920b5ae4b138b48cffd35945f12da9f9bf91fb3a4b3: Status 404 returned error can't find the container with id 532a8f698ccce99cfcc28920b5ae4b138b48cffd35945f12da9f9bf91fb3a4b3 Jul 02 22:06:26.310896 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:26.310859 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqlzp" event={"ID":"4a4ca393-cf0e-4866-b73a-76ad7790a42d","Type":"ContainerStarted","Data":"532a8f698ccce99cfcc28920b5ae4b138b48cffd35945f12da9f9bf91fb3a4b3"} Jul 02 22:06:26.315404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:26.315375 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:06:26.363267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:26.362778 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:26.640033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:26.639891 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:27.645477 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:27.645449 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:28.322247 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:28.322207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqlzp" event={"ID":"4a4ca393-cf0e-4866-b73a-76ad7790a42d","Type":"ContainerStarted","Data":"3da14109ca29783d0ca1ca9e949b3b78c074e9355b4411cc34a3635c0732f809"} Jul 02 22:06:28.322451 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:28.322276 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqlzp" event={"ID":"4a4ca393-cf0e-4866-b73a-76ad7790a42d","Type":"ContainerStarted","Data":"6f4dc606b09dc6be1767d03664c0e822de920fc8aab9617f372ca62c888dab69"} Jul 02 22:06:28.339801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:28.339716 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sqlzp" podStartSLOduration=130.591469816 podStartE2EDuration="2m12.33969617s" totalImagesPullingTime="1.748226354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:28.338677081 +0000 UTC m=+133.322889162" watchObservedRunningTime="2026-07-02 22:06:28.33969617 +0000 UTC m=+133.323908250" Jul 02 22:06:28.432341 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:28.432282 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:28.494509 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:28.494473 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:31.001440 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.001395 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" podUID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" containerName="registry" containerID="cri-o://7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218" gracePeriod=30 Jul 02 22:06:31.316562 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.316524 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:06:31.341806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.341681 2583 generic.go:356] "Generic (PLEG): container finished" podID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" containerID="7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218" exitCode=0 Jul 02 22:06:31.342126 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.341788 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" Jul 02 22:06:31.342252 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.341787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" event={"ID":"d4c3bd6c-f718-468e-965a-a0dcf457c8df","Type":"ContainerDied","Data":"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218"} Jul 02 22:06:31.342339 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.342286 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b967b955b-cbb9k" event={"ID":"d4c3bd6c-f718-468e-965a-a0dcf457c8df","Type":"ContainerDied","Data":"9d2d61ce2fd7e7bca770303c504460b54bd3afa2015ea8f4c28dcfb91e75cbb9"} Jul 02 22:06:31.342445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.342345 2583 scope.go:117] "RemoveContainer" containerID="7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218" Jul 02 22:06:31.353081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.353053 2583 scope.go:117] "RemoveContainer" containerID="7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218" Jul 02 22:06:31.353490 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:06:31.353457 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218\": container with ID starting with 7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218 not found: ID does not exist" containerID="7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218" Jul 02 22:06:31.353589 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.353504 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218"} err="failed to get container status \"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218\": rpc error: code = NotFound desc = could not find container \"7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218\": container with ID starting with 7b012172e040302215776860475125de425d7344cf62c3be229449d811b49218 not found: ID does not exist" Jul 02 22:06:31.373847 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.373814 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.373847 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.373858 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.373884 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.373917 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49twd\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.373974 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.374016 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374274 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.374111 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.374274 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.374144 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration\") pod \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\" (UID: \"d4c3bd6c-f718-468e-965a-a0dcf457c8df\") " Jul 02 22:06:31.375327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.374473 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:31.375327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.375266 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:31.379804 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.379746 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:31.380153 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.380112 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd" (OuterVolumeSpecName: "kube-api-access-49twd") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "kube-api-access-49twd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:31.380383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.380356 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:06:31.380597 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.380569 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:06:31.382151 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.382130 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:31.385896 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.385867 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d4c3bd6c-f718-468e-965a-a0dcf457c8df" (UID: "d4c3bd6c-f718-468e-965a-a0dcf457c8df"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:31.475337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475281 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-installation-pull-secrets\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475329 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4c3bd6c-f718-468e-965a-a0dcf457c8df-image-registry-private-configuration\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475341 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-bound-sa-token\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475629 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475381 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4c3bd6c-f718-468e-965a-a0dcf457c8df-ca-trust-extracted\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475629 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475395 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-certificates\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475629 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475410 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49twd\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-kube-api-access-49twd\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475629 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475423 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4c3bd6c-f718-468e-965a-a0dcf457c8df-registry-tls\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.475629 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.475435 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4c3bd6c-f718-468e-965a-a0dcf457c8df-trusted-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:31.660423 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.660384 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:06:31.664482 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:31.664457 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b967b955b-cbb9k"] Jul 02 22:06:33.600368 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:33.600340 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" path="/var/lib/kubelet/pods/d4c3bd6c-f718-468e-965a-a0dcf457c8df/volumes" Jul 02 22:06:33.791885 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:33.791845 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:33.893696 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:33.893606 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:34.356312 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:34.356276 2583 generic.go:356] "Generic (PLEG): container finished" podID="b86f8bb3-61ae-4735-97b2-9d11dec6e535" containerID="42abe870d15b163cbddb946e6b3b214592c999997ca8de52144bf2f2d4969797" exitCode=0 Jul 02 22:06:34.356554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:34.356365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" event={"ID":"b86f8bb3-61ae-4735-97b2-9d11dec6e535","Type":"ContainerDied","Data":"42abe870d15b163cbddb946e6b3b214592c999997ca8de52144bf2f2d4969797"} Jul 02 22:06:34.356795 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:34.356778 2583 scope.go:117] "RemoveContainer" containerID="42abe870d15b163cbddb946e6b3b214592c999997ca8de52144bf2f2d4969797" Jul 02 22:06:35.369387 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:35.369353 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7wbrs" event={"ID":"b86f8bb3-61ae-4735-97b2-9d11dec6e535","Type":"ContainerStarted","Data":"d3b5024bdd2e025b6aed41aff6e1a3d659fb805ae444b9813c70a4c2ea6b4eec"} Jul 02 22:06:37.378710 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:37.378676 2583 generic.go:356] "Generic (PLEG): container finished" podID="c32d31b2-74db-42d9-adfe-24d20e516268" containerID="4113fc28ce597cee6847e4915fe57860552a7dd3253454cce0637115eeae0ba6" exitCode=0 Jul 02 22:06:37.379077 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:37.378742 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" event={"ID":"c32d31b2-74db-42d9-adfe-24d20e516268","Type":"ContainerDied","Data":"4113fc28ce597cee6847e4915fe57860552a7dd3253454cce0637115eeae0ba6"} Jul 02 22:06:37.379121 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:37.379084 2583 scope.go:117] "RemoveContainer" containerID="4113fc28ce597cee6847e4915fe57860552a7dd3253454cce0637115eeae0ba6" Jul 02 22:06:37.943066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:37.943032 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d48d4fd89-qjr8q_bd81f4e8-4237-4f44-a4ea-8886d1bf8441/router/0.log" Jul 02 22:06:37.960065 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:37.960038 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxxn5_ac9ca684-7062-49c0-bbaf-5dd463e3ecfb/serve-healthcheck-canary/0.log" Jul 02 22:06:38.384721 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:38.384617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-srrnc" event={"ID":"c32d31b2-74db-42d9-adfe-24d20e516268","Type":"ContainerStarted","Data":"390c61e3d9821b6312368dbcc49a73101e8a61f8019f5b83a10120f43944bf04"} Jul 02 22:06:40.870880 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:40.870841 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:40.975017 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:40.974971 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:41.039555 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:41.039521 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:41.122794 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:41.122718 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:41.127260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:41.127236 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76958644b7-8rnf7" Jul 02 22:06:41.397905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:41.397821 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:42.402138 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:42.402051 2583 generic.go:356] "Generic (PLEG): container finished" podID="9d469a43-f522-4f7c-b597-2ae7c3bff1db" containerID="8456ba213e893097844f59492d481b6e20f79f27b1e80da0fd44ea4f53c94507" exitCode=0 Jul 02 22:06:42.402138 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:42.402104 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" event={"ID":"9d469a43-f522-4f7c-b597-2ae7c3bff1db","Type":"ContainerDied","Data":"8456ba213e893097844f59492d481b6e20f79f27b1e80da0fd44ea4f53c94507"} Jul 02 22:06:42.402590 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:42.402477 2583 scope.go:117] "RemoveContainer" containerID="8456ba213e893097844f59492d481b6e20f79f27b1e80da0fd44ea4f53c94507" Jul 02 22:06:43.408467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:43.408433 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-gcwlb" event={"ID":"9d469a43-f522-4f7c-b597-2ae7c3bff1db","Type":"ContainerStarted","Data":"b29f68b8bb8bef9c63897a12985839021e0cb06d669e18306051701e6b8bbd39"} Jul 02 22:06:46.286479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:46.286447 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:48.680414 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:48.680375 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:48.738283 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:48.738246 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:51.394988 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.394947 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d44b866bc-qkhtx" podUID="b55ff76a-e01c-4945-9d36-bce93fd917a4" containerName="console" containerID="cri-o://f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b" gracePeriod=15 Jul 02 22:06:51.714017 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.713662 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:51.864662 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864627 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.864662 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864673 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.864878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864727 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.864878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864752 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8ftq\" (UniqueName: \"kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.864878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864777 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.865024 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.864897 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config\") pod \"b55ff76a-e01c-4945-9d36-bce93fd917a4\" (UID: \"b55ff76a-e01c-4945-9d36-bce93fd917a4\") " Jul 02 22:06:51.865086 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.865039 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:51.865288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.865145 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-oauth-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:51.865288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.865222 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca" (OuterVolumeSpecName: "service-ca") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:51.865467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.865375 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config" (OuterVolumeSpecName: "console-config") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:51.867244 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.867217 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:06:51.867347 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.867232 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq" (OuterVolumeSpecName: "kube-api-access-v8ftq") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "kube-api-access-v8ftq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:51.867347 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.867289 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b55ff76a-e01c-4945-9d36-bce93fd917a4" (UID: "b55ff76a-e01c-4945-9d36-bce93fd917a4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:06:51.966139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.966102 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-oauth-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:51.966139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.966135 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:51.966139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.966145 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55ff76a-e01c-4945-9d36-bce93fd917a4-console-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:51.966389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.966153 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8ftq\" (UniqueName: \"kubernetes.io/projected/b55ff76a-e01c-4945-9d36-bce93fd917a4-kube-api-access-v8ftq\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:51.966389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:51.966162 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ff76a-e01c-4945-9d36-bce93fd917a4-service-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.454749 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.454716 2583 generic.go:356] "Generic (PLEG): container finished" podID="b55ff76a-e01c-4945-9d36-bce93fd917a4" containerID="f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b" exitCode=0 Jul 02 22:06:52.455343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.454759 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d44b866bc-qkhtx" Jul 02 22:06:52.455343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.454805 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d44b866bc-qkhtx" event={"ID":"b55ff76a-e01c-4945-9d36-bce93fd917a4","Type":"ContainerDied","Data":"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b"} Jul 02 22:06:52.455343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.454858 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d44b866bc-qkhtx" event={"ID":"b55ff76a-e01c-4945-9d36-bce93fd917a4","Type":"ContainerDied","Data":"c116a4ccd71d638973f92bdc0b7826c44d000829151c6a691ba764f9ef766e2c"} Jul 02 22:06:52.455343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.454882 2583 scope.go:117] "RemoveContainer" containerID="f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b" Jul 02 22:06:52.463803 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.463784 2583 scope.go:117] "RemoveContainer" containerID="f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b" Jul 02 22:06:52.464078 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:06:52.464061 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b\": container with ID starting with f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b not found: ID does not exist" containerID="f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b" Jul 02 22:06:52.464143 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.464090 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b"} err="failed to get container status \"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b\": rpc error: code = NotFound desc = could not find container \"f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b\": container with ID starting with f8259eb41328950987749d132359e7394250ead20bc7d51f90e09ee9db6c362b not found: ID does not exist" Jul 02 22:06:52.477094 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.477058 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:52.478470 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:52.478451 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d44b866bc-qkhtx"] Jul 02 22:06:53.601087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:06:53.601044 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55ff76a-e01c-4945-9d36-bce93fd917a4" path="/var/lib/kubelet/pods/b55ff76a-e01c-4945-9d36-bce93fd917a4/volumes" Jul 02 22:07:15.754389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:15.754287 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:16.772382 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.772333 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:16.796352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796295 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:16.796666 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796643 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="alertmanager" containerID="cri-o://0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" gracePeriod=120 Jul 02 22:07:16.796733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796702 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-metric" containerID="cri-o://d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" gracePeriod=120 Jul 02 22:07:16.796792 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796743 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy" containerID="cri-o://daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" gracePeriod=120 Jul 02 22:07:16.796843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796774 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="prom-label-proxy" containerID="cri-o://9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" gracePeriod=120 Jul 02 22:07:16.796843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796808 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="config-reloader" containerID="cri-o://4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" gracePeriod=120 Jul 02 22:07:16.796950 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:16.796731 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-web" containerID="cri-o://1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" gracePeriod=120 Jul 02 22:07:17.555114 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555079 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" exitCode=0 Jul 02 22:07:17.555114 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555107 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" exitCode=0 Jul 02 22:07:17.555114 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555116 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" exitCode=0 Jul 02 22:07:17.555114 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555122 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" exitCode=0 Jul 02 22:07:17.555426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555151 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d"} Jul 02 22:07:17.555426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a"} Jul 02 22:07:17.555426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555240 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666"} Jul 02 22:07:17.555426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:17.555266 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139"} Jul 02 22:07:18.047160 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.047135 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.066397 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066364 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066408 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066439 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066469 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066496 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066525 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066613 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066666 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066719 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066767 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.066818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066797 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.067054 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066830 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xzth\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.067054 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066879 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca\") pod \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\" (UID: \"b8b6928a-bb3b-4702-8944-c1224c2eeb26\") " Jul 02 22:07:18.067054 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.066942 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:07:18.067213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.067056 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:07:18.067213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.067179 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.067213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.067199 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-alertmanager-main-db\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.067747 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.067692 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:07:18.069692 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.069662 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.069831 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.069754 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.070072 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.070034 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.070364 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.070336 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:07:18.070568 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.070546 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out" (OuterVolumeSpecName: "config-out") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:07:18.070681 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.070660 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume" (OuterVolumeSpecName: "config-volume") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.070998 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.070972 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.071155 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.071138 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth" (OuterVolumeSpecName: "kube-api-access-2xzth") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "kube-api-access-2xzth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:07:18.074705 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.074528 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.082436 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.082407 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config" (OuterVolumeSpecName: "web-config") pod "b8b6928a-bb3b-4702-8944-c1224c2eeb26" (UID: "b8b6928a-bb3b-4702-8944-c1224c2eeb26"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167588 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-main-tls\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167622 2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-cluster-tls-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167633 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167643 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167653 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-web-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167663 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-tls-assets\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167670 2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-volume\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167679 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8b6928a-bb3b-4702-8944-c1224c2eeb26-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167690 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b8b6928a-bb3b-4702-8944-c1224c2eeb26-config-out\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.167688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167698 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xzth\" (UniqueName: \"kubernetes.io/projected/b8b6928a-bb3b-4702-8944-c1224c2eeb26-kube-api-access-2xzth\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.168123 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.167707 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8b6928a-bb3b-4702-8944-c1224c2eeb26-metrics-client-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:07:18.562454 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562417 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" exitCode=0 Jul 02 22:07:18.562454 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562445 2583 generic.go:356] "Generic (PLEG): container finished" podID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerID="1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" exitCode=0 Jul 02 22:07:18.562690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562504 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.562690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3"} Jul 02 22:07:18.562690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562642 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897"} Jul 02 22:07:18.562690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562671 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b8b6928a-bb3b-4702-8944-c1224c2eeb26","Type":"ContainerDied","Data":"ceb28ece64115d9c9a089664d81196df6e357cd23cfc9487f246cb5c87ea3f0f"} Jul 02 22:07:18.562806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.562702 2583 scope.go:117] "RemoveContainer" containerID="9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" Jul 02 22:07:18.570924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.570902 2583 scope.go:117] "RemoveContainer" containerID="d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" Jul 02 22:07:18.578699 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.578680 2583 scope.go:117] "RemoveContainer" containerID="daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" Jul 02 22:07:18.586451 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.586319 2583 scope.go:117] "RemoveContainer" containerID="1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" Jul 02 22:07:18.588694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.588669 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:18.593011 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.592987 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:18.594971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.594949 2583 scope.go:117] "RemoveContainer" containerID="4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" Jul 02 22:07:18.602714 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.602693 2583 scope.go:117] "RemoveContainer" containerID="0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" Jul 02 22:07:18.610258 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.610236 2583 scope.go:117] "RemoveContainer" containerID="30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc" Jul 02 22:07:18.617075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617046 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:18.617359 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617345 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="config-reloader" Jul 02 22:07:18.617359 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617361 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="config-reloader" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617374 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617381 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617389 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="alertmanager" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617394 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="alertmanager" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617402 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="init-config-reloader" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617408 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="init-config-reloader" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617414 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="prom-label-proxy" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617421 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="prom-label-proxy" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617426 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55ff76a-e01c-4945-9d36-bce93fd917a4" containerName="console" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617430 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55ff76a-e01c-4945-9d36-bce93fd917a4" containerName="console" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617437 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-metric" Jul 02 22:07:18.617443 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617442 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-metric" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617452 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" containerName="registry" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617457 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" containerName="registry" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617469 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-web" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617474 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-web" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617532 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="alertmanager" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617541 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-web" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617549 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="config-reloader" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617557 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="prom-label-proxy" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617563 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55ff76a-e01c-4945-9d36-bce93fd917a4" containerName="console" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617569 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c3bd6c-f718-468e-965a-a0dcf457c8df" containerName="registry" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617576 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy-metric" Jul 02 22:07:18.617789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.617582 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" containerName="kube-rbac-proxy" Jul 02 22:07:18.618189 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.618147 2583 scope.go:117] "RemoveContainer" containerID="9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" Jul 02 22:07:18.618504 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.618477 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d\": container with ID starting with 9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d not found: ID does not exist" containerID="9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" Jul 02 22:07:18.618623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.618513 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d"} err="failed to get container status \"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d\": rpc error: code = NotFound desc = could not find container \"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d\": container with ID starting with 9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d not found: ID does not exist" Jul 02 22:07:18.618623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.618537 2583 scope.go:117] "RemoveContainer" containerID="d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" Jul 02 22:07:18.618803 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.618778 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3\": container with ID starting with d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3 not found: ID does not exist" containerID="d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" Jul 02 22:07:18.618849 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.618814 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3"} err="failed to get container status \"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3\": rpc error: code = NotFound desc = could not find container \"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3\": container with ID starting with d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3 not found: ID does not exist" Jul 02 22:07:18.618849 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.618838 2583 scope.go:117] "RemoveContainer" containerID="daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" Jul 02 22:07:18.619099 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.619082 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a\": container with ID starting with daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a not found: ID does not exist" containerID="daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" Jul 02 22:07:18.619170 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619105 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a"} err="failed to get container status \"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a\": rpc error: code = NotFound desc = could not find container \"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a\": container with ID starting with daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a not found: ID does not exist" Jul 02 22:07:18.619170 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619126 2583 scope.go:117] "RemoveContainer" containerID="1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" Jul 02 22:07:18.619422 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.619405 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897\": container with ID starting with 1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897 not found: ID does not exist" containerID="1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" Jul 02 22:07:18.619496 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619429 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897"} err="failed to get container status \"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897\": rpc error: code = NotFound desc = could not find container \"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897\": container with ID starting with 1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897 not found: ID does not exist" Jul 02 22:07:18.619496 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619449 2583 scope.go:117] "RemoveContainer" containerID="4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" Jul 02 22:07:18.619705 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.619688 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666\": container with ID starting with 4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666 not found: ID does not exist" containerID="4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" Jul 02 22:07:18.619748 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619709 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666"} err="failed to get container status \"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666\": rpc error: code = NotFound desc = could not find container \"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666\": container with ID starting with 4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666 not found: ID does not exist" Jul 02 22:07:18.619748 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619724 2583 scope.go:117] "RemoveContainer" containerID="0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" Jul 02 22:07:18.619964 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.619942 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139\": container with ID starting with 0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139 not found: ID does not exist" containerID="0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" Jul 02 22:07:18.620005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619969 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139"} err="failed to get container status \"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139\": rpc error: code = NotFound desc = could not find container \"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139\": container with ID starting with 0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139 not found: ID does not exist" Jul 02 22:07:18.620005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.619982 2583 scope.go:117] "RemoveContainer" containerID="30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc" Jul 02 22:07:18.620190 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:07:18.620173 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc\": container with ID starting with 30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc not found: ID does not exist" containerID="30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc" Jul 02 22:07:18.620228 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620194 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc"} err="failed to get container status \"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc\": rpc error: code = NotFound desc = could not find container \"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc\": container with ID starting with 30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc not found: ID does not exist" Jul 02 22:07:18.620228 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620206 2583 scope.go:117] "RemoveContainer" containerID="9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d" Jul 02 22:07:18.620452 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620425 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d"} err="failed to get container status \"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d\": rpc error: code = NotFound desc = could not find container \"9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d\": container with ID starting with 9efcb7c621bf391760ff3bd0f6b4827a0c9fed4c61cf24886632fc53a736e90d not found: ID does not exist" Jul 02 22:07:18.620452 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620448 2583 scope.go:117] "RemoveContainer" containerID="d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3" Jul 02 22:07:18.620626 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620610 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.620711 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620692 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3"} err="failed to get container status \"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3\": rpc error: code = NotFound desc = could not find container \"d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3\": container with ID starting with d9562d995500b0b92b34b20319a353019e53773e3d0474c16bca0cdd52a2e3b3 not found: ID does not exist" Jul 02 22:07:18.620764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620712 2583 scope.go:117] "RemoveContainer" containerID="daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a" Jul 02 22:07:18.620983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620947 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a"} err="failed to get container status \"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a\": rpc error: code = NotFound desc = could not find container \"daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a\": container with ID starting with daa1c9d729f1392532b600c282c18fc97665f1d69967b707256b7a1017c2a10a not found: ID does not exist" Jul 02 22:07:18.620983 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.620971 2583 scope.go:117] "RemoveContainer" containerID="1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897" Jul 02 22:07:18.621215 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621192 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897"} err="failed to get container status \"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897\": rpc error: code = NotFound desc = could not find container \"1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897\": container with ID starting with 1bb247554c3be89c0ac1e441746871b20aa19e83662a45094ab3584270756897 not found: ID does not exist" Jul 02 22:07:18.621215 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621213 2583 scope.go:117] "RemoveContainer" containerID="4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666" Jul 02 22:07:18.621498 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621479 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666"} err="failed to get container status \"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666\": rpc error: code = NotFound desc = could not find container \"4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666\": container with ID starting with 4d43b46e5ab71ff0a141b00d00ab7ee59f82ab7081aa67188b42cf7965d0a666 not found: ID does not exist" Jul 02 22:07:18.621558 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621502 2583 scope.go:117] "RemoveContainer" containerID="0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139" Jul 02 22:07:18.621755 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621738 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139"} err="failed to get container status \"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139\": rpc error: code = NotFound desc = could not find container \"0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139\": container with ID starting with 0435d0c67960e1d6894f89ef3896dda6cbdbfcba2b266fba958feceed52ba139 not found: ID does not exist" Jul 02 22:07:18.621755 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621754 2583 scope.go:117] "RemoveContainer" containerID="30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc" Jul 02 22:07:18.621959 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.621932 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc"} err="failed to get container status \"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc\": rpc error: code = NotFound desc = could not find container \"30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc\": container with ID starting with 30884f7c9e1bc5980c00532094cfff5ea62b454f7749726daddc307bac6a0bbc not found: ID does not exist" Jul 02 22:07:18.623557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623536 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 22:07:18.623633 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623576 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 22:07:18.623740 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623724 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 22:07:18.623804 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623793 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fprrc\"" Jul 02 22:07:18.623851 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623808 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 22:07:18.623851 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623818 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 22:07:18.623851 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.623817 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 22:07:18.624265 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.624251 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 22:07:18.624323 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.624270 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 22:07:18.630041 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.630018 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 22:07:18.640918 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.640893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:18.670980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.670943 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.670980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.670981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-web-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671087 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-config-out\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671147 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpflm\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-kube-api-access-cpflm\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671246 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671269 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671294 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671347 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671377 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.671401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.671395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772444 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772444 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772447 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772498 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-web-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-config-out\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772778 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772808 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772825 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpflm\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-kube-api-access-cpflm\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.772878 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.773194 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.772884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.774033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.773696 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.774033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.773769 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775596 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775752 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775711 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775752 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775740 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-web-config\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775882 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775973 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775887 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.775973 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775956 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.776079 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.775976 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.776565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.776543 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.776888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.776869 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a303012c-f3ed-42de-803b-794a64f4248b-config-volume\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.777497 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.777473 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a303012c-f3ed-42de-803b-794a64f4248b-config-out\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.784252 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.784233 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpflm\" (UniqueName: \"kubernetes.io/projected/a303012c-f3ed-42de-803b-794a64f4248b-kube-api-access-cpflm\") pod \"alertmanager-main-0\" (UID: \"a303012c-f3ed-42de-803b-794a64f4248b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:18.931491 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:18.931447 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 22:07:19.098506 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:19.098481 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 22:07:19.100493 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:07:19.100464 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda303012c_f3ed_42de_803b_794a64f4248b.slice/crio-a424cd23e7a1e75091bfcda91c3977e4af2ca85ea359950818a5d908877ad431 WatchSource:0}: Error finding container a424cd23e7a1e75091bfcda91c3977e4af2ca85ea359950818a5d908877ad431: Status 404 returned error can't find the container with id a424cd23e7a1e75091bfcda91c3977e4af2ca85ea359950818a5d908877ad431 Jul 02 22:07:19.568080 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:19.568046 2583 generic.go:356] "Generic (PLEG): container finished" podID="a303012c-f3ed-42de-803b-794a64f4248b" containerID="3d047ded9e597f4d96fe411333af065d6e186bd44ee66695e40c1cd9b66560bd" exitCode=0 Jul 02 22:07:19.568240 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:19.568142 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerDied","Data":"3d047ded9e597f4d96fe411333af065d6e186bd44ee66695e40c1cd9b66560bd"} Jul 02 22:07:19.568240 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:19.568229 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"a424cd23e7a1e75091bfcda91c3977e4af2ca85ea359950818a5d908877ad431"} Jul 02 22:07:19.600231 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:19.600203 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b6928a-bb3b-4702-8944-c1224c2eeb26" path="/var/lib/kubelet/pods/b8b6928a-bb3b-4702-8944-c1224c2eeb26/volumes" Jul 02 22:07:20.576747 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576706 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"6ee50517afa01172279f1ebb717c6b580060c20cb79c0c6bcf5c73b38082f0e3"} Jul 02 22:07:20.577190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576774 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"a201f1c374ccd684f9d0c9f4568593bf84fdc2afcfea09a61803681bb2dcd19c"} Jul 02 22:07:20.577190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576801 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"e50ef5707f55153f5a0d19f694ad3f303be64a26796d098770c3cba3a690749f"} Jul 02 22:07:20.577190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576833 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"ea0feba9d8757e85bfe8af43fa629770f89ffaa640bb93f1544531309f7339d5"} Jul 02 22:07:20.577190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576858 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"45459fa6c173e29f1009818cefdde1be9d7cee4f1de5f6a6905cd7a795195364"} Jul 02 22:07:20.577190 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.576889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a303012c-f3ed-42de-803b-794a64f4248b","Type":"ContainerStarted","Data":"a5cd5c2f465b15af6134070fcb3f1790d2664bd29db4bc1b9a55ac21aaa3ce94"} Jul 02 22:07:20.605107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.605048 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.58766068 podStartE2EDuration="2.605032157s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.371477ms" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:20.603428108 +0000 UTC m=+185.587640190" watchObservedRunningTime="2026-07-02 22:07:20.605032157 +0000 UTC m=+185.589244270" Jul 02 22:07:20.813670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.813632 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-546c5f6498-zlzww"] Jul 02 22:07:20.816083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.816063 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.819516 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.819487 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 22:07:20.819653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.819490 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 22:07:20.819653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.819549 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 22:07:20.819653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.819635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xvzpg\"" Jul 02 22:07:20.819921 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.819905 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 22:07:20.820077 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.820060 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 22:07:20.826155 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.826133 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 22:07:20.832496 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.832427 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546c5f6498-zlzww"] Jul 02 22:07:20.895650 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895603 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895650 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-federate-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895731 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-metrics-client-ca\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.895872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895866 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wvx\" (UniqueName: \"kubernetes.io/projected/322e8ba0-a95f-4079-9cdc-28f1acd92284-kube-api-access-p5wvx\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.896093 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.895965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-serving-certs-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997048 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-metrics-client-ca\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997062 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997116 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wvx\" (UniqueName: \"kubernetes.io/projected/322e8ba0-a95f-4079-9cdc-28f1acd92284-kube-api-access-p5wvx\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-serving-certs-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997211 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997237 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-federate-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.997980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997947 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-serving-certs-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.998120 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.997954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-metrics-client-ca\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.998385 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.998363 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:20.999978 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:20.999948 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-federate-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.000086 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.000026 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.000194 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.000176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-telemeter-client-tls\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.000343 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.000321 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/322e8ba0-a95f-4079-9cdc-28f1acd92284-secret-telemeter-client\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.006034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.006014 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wvx\" (UniqueName: \"kubernetes.io/projected/322e8ba0-a95f-4079-9cdc-28f1acd92284-kube-api-access-p5wvx\") pod \"telemeter-client-546c5f6498-zlzww\" (UID: \"322e8ba0-a95f-4079-9cdc-28f1acd92284\") " pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.126753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.126654 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" Jul 02 22:07:21.265456 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.265431 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546c5f6498-zlzww"] Jul 02 22:07:21.267254 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:07:21.267227 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322e8ba0_a95f_4079_9cdc_28f1acd92284.slice/crio-380e8d8b9555558e0b75f0ea5b41221c628025e9db7726fdcf7b422e1397edce WatchSource:0}: Error finding container 380e8d8b9555558e0b75f0ea5b41221c628025e9db7726fdcf7b422e1397edce: Status 404 returned error can't find the container with id 380e8d8b9555558e0b75f0ea5b41221c628025e9db7726fdcf7b422e1397edce Jul 02 22:07:21.582648 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:21.582613 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" event={"ID":"322e8ba0-a95f-4079-9cdc-28f1acd92284","Type":"ContainerStarted","Data":"380e8d8b9555558e0b75f0ea5b41221c628025e9db7726fdcf7b422e1397edce"} Jul 02 22:07:23.593980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:23.593879 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" event={"ID":"322e8ba0-a95f-4079-9cdc-28f1acd92284","Type":"ContainerStarted","Data":"90aab5cccc3e9c96f0a308398a58bec87dac436446061f555b6fcc4e6f112c1a"} Jul 02 22:07:23.593980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:23.593948 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" event={"ID":"322e8ba0-a95f-4079-9cdc-28f1acd92284","Type":"ContainerStarted","Data":"ee5db712f0f6f2ceb92a05d1e6f76dd4621f63d9b7da1e6fa858e6ab84e56272"} Jul 02 22:07:23.593980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:23.593978 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" event={"ID":"322e8ba0-a95f-4079-9cdc-28f1acd92284","Type":"ContainerStarted","Data":"46e7e358a3998081e90c24d223a12d25fc2d0a945f3b7cc561c8404fb76d5df3"} Jul 02 22:07:23.619132 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:23.619079 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-546c5f6498-zlzww" podStartSLOduration=1.674915106 podStartE2EDuration="3.619062978s" totalImagesPullingTime="1.944147872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:23.617637936 +0000 UTC m=+188.601850016" watchObservedRunningTime="2026-07-02 22:07:23.619062978 +0000 UTC m=+188.603275061" Jul 02 22:07:24.378116 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.378080 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:07:24.380881 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.380847 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.393971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.393933 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:07:24.431076 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431039 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlgb6\" (UniqueName: \"kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431507 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431266 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431507 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431335 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.431507 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.431391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.435117 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.435077 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:24.490125 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.490082 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:24.532257 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532340 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlgb6\" (UniqueName: \"kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532607 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532607 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532511 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.532607 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.532557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.533141 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.533114 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.533277 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.533175 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.533277 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.533243 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.533277 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.533243 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.534987 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.534946 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.535172 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.535102 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.541197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.541171 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlgb6\" (UniqueName: \"kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6\") pod \"console-99d4c885d-42h27\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.570900 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.570858 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:24.691539 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.691496 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:24.832488 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:24.832440 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:07:24.834649 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:07:24.834615 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0a26d6_f7c9_4789_bad6_a3798eb69013.slice/crio-90d66020baab8ee8f85eff645bb9c89a36e9c5a2dc9d49b8e33b8d668115bde5 WatchSource:0}: Error finding container 90d66020baab8ee8f85eff645bb9c89a36e9c5a2dc9d49b8e33b8d668115bde5: Status 404 returned error can't find the container with id 90d66020baab8ee8f85eff645bb9c89a36e9c5a2dc9d49b8e33b8d668115bde5 Jul 02 22:07:25.171741 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:25.171646 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:25.230875 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:25.230838 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:25.607968 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:25.607928 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-99d4c885d-42h27" event={"ID":"8e0a26d6-f7c9-4789-bad6-a3798eb69013","Type":"ContainerStarted","Data":"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce"} Jul 02 22:07:25.608173 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:25.607981 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-99d4c885d-42h27" event={"ID":"8e0a26d6-f7c9-4789-bad6-a3798eb69013","Type":"ContainerStarted","Data":"90d66020baab8ee8f85eff645bb9c89a36e9c5a2dc9d49b8e33b8d668115bde5"} Jul 02 22:07:34.692195 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:34.692154 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:34.692666 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:34.692251 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:34.697282 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:34.697256 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:34.715734 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:34.715681 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-99d4c885d-42h27" podStartSLOduration=10.715661988 podStartE2EDuration="10.715661988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:25.627487028 +0000 UTC m=+190.611699113" watchObservedRunningTime="2026-07-02 22:07:34.715661988 +0000 UTC m=+199.699874070" Jul 02 22:07:35.653922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:35.653893 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:07:35.702185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:35.702152 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:07:35.816034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:35.815996 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:35.917067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:35.916972 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:36.017273 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:07:36.017237 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:00.727418 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:00.727377 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7766fb46b5-85lnf" podUID="f6beb7a4-c640-4301-acba-afc0396881ae" containerName="console" containerID="cri-o://ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e" gracePeriod=15 Jul 02 22:08:00.980981 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:00.980914 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:08:01.051974 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.051928 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052175 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.051985 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052175 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052025 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052175 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052075 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052378 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052200 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c85mx\" (UniqueName: \"kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052378 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052295 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052378 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052367 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config\") pod \"f6beb7a4-c640-4301-acba-afc0396881ae\" (UID: \"f6beb7a4-c640-4301-acba-afc0396881ae\") " Jul 02 22:08:01.052521 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052416 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:08:01.052521 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052467 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:08:01.052631 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052513 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:08:01.052712 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052687 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config" (OuterVolumeSpecName: "console-config") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:08:01.052776 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052719 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-oauth-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.052776 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052739 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-service-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.052776 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.052755 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-trusted-ca-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.054327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.054287 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx" (OuterVolumeSpecName: "kube-api-access-c85mx") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "kube-api-access-c85mx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:08:01.054327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.054315 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:08:01.054486 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.054359 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f6beb7a4-c640-4301-acba-afc0396881ae" (UID: "f6beb7a4-c640-4301-acba-afc0396881ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:08:01.153441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.153399 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.153441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.153430 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c85mx\" (UniqueName: \"kubernetes.io/projected/f6beb7a4-c640-4301-acba-afc0396881ae-kube-api-access-c85mx\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.153441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.153440 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6beb7a4-c640-4301-acba-afc0396881ae-console-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.153441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.153449 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6beb7a4-c640-4301-acba-afc0396881ae-console-oauth-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:08:01.766730 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.766697 2583 generic.go:356] "Generic (PLEG): container finished" podID="f6beb7a4-c640-4301-acba-afc0396881ae" containerID="ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e" exitCode=0 Jul 02 22:08:01.767183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.766747 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766fb46b5-85lnf" Jul 02 22:08:01.767183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.766775 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766fb46b5-85lnf" event={"ID":"f6beb7a4-c640-4301-acba-afc0396881ae","Type":"ContainerDied","Data":"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e"} Jul 02 22:08:01.767183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.766816 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766fb46b5-85lnf" event={"ID":"f6beb7a4-c640-4301-acba-afc0396881ae","Type":"ContainerDied","Data":"e6e4b2ab2293e940d61c28fc43f38857883700c3019c688f73b4888eb0289c14"} Jul 02 22:08:01.767183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.766836 2583 scope.go:117] "RemoveContainer" containerID="ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e" Jul 02 22:08:01.775416 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.775362 2583 scope.go:117] "RemoveContainer" containerID="ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e" Jul 02 22:08:01.775690 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:08:01.775667 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e\": container with ID starting with ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e not found: ID does not exist" containerID="ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e" Jul 02 22:08:01.775735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.775701 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e"} err="failed to get container status \"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e\": rpc error: code = NotFound desc = could not find container \"ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e\": container with ID starting with ead008b358e52307cafb15391ab892f32e240bfca84c567b27dc5cd0f14c0b3e not found: ID does not exist" Jul 02 22:08:01.783695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.783666 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:08:01.786279 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:01.786256 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7766fb46b5-85lnf"] Jul 02 22:08:03.600382 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:03.600352 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6beb7a4-c640-4301-acba-afc0396881ae" path="/var/lib/kubelet/pods/f6beb7a4-c640-4301-acba-afc0396881ae/volumes" Jul 02 22:08:08.148886 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:08.148849 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:08.202178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:08.202145 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:30.990380 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:30.990345 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:31.063980 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:31.063943 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:42.732616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.732529 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:08:42.733034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.732881 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6beb7a4-c640-4301-acba-afc0396881ae" containerName="console" Jul 02 22:08:42.733034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.732897 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6beb7a4-c640-4301-acba-afc0396881ae" containerName="console" Jul 02 22:08:42.733034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.732970 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6beb7a4-c640-4301-acba-afc0396881ae" containerName="console" Jul 02 22:08:42.735810 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.735789 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.748856 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.748828 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:08:42.795522 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.795484 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:42.812591 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812554 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.812764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.812764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.812764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.812764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.812764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.813002 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.812801 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdb7s\" (UniqueName: \"kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.897484 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.897450 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:42.913914 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.913877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914002 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.913922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914056 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914236 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914273 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdb7s\" (UniqueName: \"kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914729 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914877 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914759 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914877 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914848 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.914962 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.914939 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.917009 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.916981 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.917089 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.917073 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.925843 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.925819 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdb7s\" (UniqueName: \"kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s\") pod \"console-f469f9f4d-v4z8w\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:42.997615 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:42.997523 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:43.046778 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.046736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:43.059914 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.059878 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:43.184708 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.184667 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:08:43.187888 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:08:43.187860 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eade025_036b_48d6_bc07_82b7f0d8acf9.slice/crio-a7c780781b5ca11d4ed63cf7e36fb130ac6aa58b5d1b11780bc22dde8b8031f4 WatchSource:0}: Error finding container a7c780781b5ca11d4ed63cf7e36fb130ac6aa58b5d1b11780bc22dde8b8031f4: Status 404 returned error can't find the container with id a7c780781b5ca11d4ed63cf7e36fb130ac6aa58b5d1b11780bc22dde8b8031f4 Jul 02 22:08:43.938405 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.938361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f469f9f4d-v4z8w" event={"ID":"8eade025-036b-48d6-bc07-82b7f0d8acf9","Type":"ContainerStarted","Data":"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b"} Jul 02 22:08:43.938801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.938424 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f469f9f4d-v4z8w" event={"ID":"8eade025-036b-48d6-bc07-82b7f0d8acf9","Type":"ContainerStarted","Data":"a7c780781b5ca11d4ed63cf7e36fb130ac6aa58b5d1b11780bc22dde8b8031f4"} Jul 02 22:08:43.957582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:43.957520 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f469f9f4d-v4z8w" podStartSLOduration=1.957504216 podStartE2EDuration="1.957504216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:08:43.955872574 +0000 UTC m=+268.940084669" watchObservedRunningTime="2026-07-02 22:08:43.957504216 +0000 UTC m=+268.941716352" Jul 02 22:08:48.568316 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:48.568266 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:48.631669 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:48.631633 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:48.706139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:48.706094 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:48.805620 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:48.805582 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:53.047041 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:53.046997 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:53.047561 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:53.047077 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:53.051954 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:53.051927 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:53.983071 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:53.983044 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:08:54.033216 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:54.033182 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:08:54.147630 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:54.147597 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:54.245314 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:54.245210 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:54.301114 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:08:54.301081 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:03.532635 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:03.532594 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:03.633387 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:03.633349 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:15.458457 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:15.458422 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:09:15.460790 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:15.460759 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:09:15.474408 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:15.474383 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 22:09:19.058895 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.058827 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-99d4c885d-42h27" podUID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" containerName="console" containerID="cri-o://d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce" gracePeriod=15 Jul 02 22:09:19.292263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.292234 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:19.312038 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.311970 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:09:19.386364 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.386293 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:19.446066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446030 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlgb6\" (UniqueName: \"kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446072 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446103 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446145 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446187 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446250 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446353 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446349 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config\") pod \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\" (UID: \"8e0a26d6-f7c9-4789-bad6-a3798eb69013\") " Jul 02 22:09:19.446718 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446684 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:19.446718 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446706 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:19.446834 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.446728 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca" (OuterVolumeSpecName: "service-ca") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:19.447055 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.447032 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config" (OuterVolumeSpecName: "console-config") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:19.448396 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.448360 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:09:19.448501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.448436 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:09:19.448501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.448450 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6" (OuterVolumeSpecName: "kube-api-access-zlgb6") pod "8e0a26d6-f7c9-4789-bad6-a3798eb69013" (UID: "8e0a26d6-f7c9-4789-bad6-a3798eb69013"). InnerVolumeSpecName "kube-api-access-zlgb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:19.547759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547720 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.547759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547751 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-oauth-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.547759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547760 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-trusted-ca-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.547759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547771 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.548013 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547780 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlgb6\" (UniqueName: \"kubernetes.io/projected/8e0a26d6-f7c9-4789-bad6-a3798eb69013-kube-api-access-zlgb6\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.548013 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547788 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0a26d6-f7c9-4789-bad6-a3798eb69013-service-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.548013 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.547796 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e0a26d6-f7c9-4789-bad6-a3798eb69013-console-oauth-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:19.936024 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.935990 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7"] Jul 02 22:09:19.936453 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.936434 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" containerName="console" Jul 02 22:09:19.936558 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.936455 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" containerName="console" Jul 02 22:09:19.936558 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.936528 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" containerName="console" Jul 02 22:09:19.939835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.939809 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:19.942752 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.942727 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 22:09:19.942866 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.942819 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kwtd8\"" Jul 02 22:09:19.942955 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.942937 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 22:09:19.948877 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:19.948840 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7"] Jul 02 22:09:20.051947 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.051914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.052137 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.051959 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.052137 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.052093 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkjgm\" (UniqueName: \"kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.088838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.088807 2583 generic.go:356] "Generic (PLEG): container finished" podID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" containerID="d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce" exitCode=0 Jul 02 22:09:20.089225 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.088854 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-99d4c885d-42h27" Jul 02 22:09:20.089225 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.088896 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-99d4c885d-42h27" event={"ID":"8e0a26d6-f7c9-4789-bad6-a3798eb69013","Type":"ContainerDied","Data":"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce"} Jul 02 22:09:20.089225 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.088948 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-99d4c885d-42h27" event={"ID":"8e0a26d6-f7c9-4789-bad6-a3798eb69013","Type":"ContainerDied","Data":"90d66020baab8ee8f85eff645bb9c89a36e9c5a2dc9d49b8e33b8d668115bde5"} Jul 02 22:09:20.089225 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.088975 2583 scope.go:117] "RemoveContainer" containerID="d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce" Jul 02 22:09:20.097410 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.097357 2583 scope.go:117] "RemoveContainer" containerID="d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce" Jul 02 22:09:20.097653 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:09:20.097635 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce\": container with ID starting with d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce not found: ID does not exist" containerID="d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce" Jul 02 22:09:20.097693 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.097661 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce"} err="failed to get container status \"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce\": rpc error: code = NotFound desc = could not find container \"d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce\": container with ID starting with d120e9d426011fcff02388fc00cc6c4d99cd0888b26abe999db48bee28903fce not found: ID does not exist" Jul 02 22:09:20.106219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.106192 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:09:20.109917 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.109882 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-99d4c885d-42h27"] Jul 02 22:09:20.152735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.152691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkjgm\" (UniqueName: \"kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.152899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.152754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.152899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.152789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.153240 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.153219 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.153330 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.153228 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.161255 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.161230 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkjgm\" (UniqueName: \"kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.250341 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.250222 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:20.386771 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.386742 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7"] Jul 02 22:09:20.389202 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:09:20.389168 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f183026_5a10_48dd_a0c3_0d0a64337812.slice/crio-c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc WatchSource:0}: Error finding container c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc: Status 404 returned error can't find the container with id c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc Jul 02 22:09:20.391205 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:20.391186 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:09:21.096324 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:21.096271 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerStarted","Data":"c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc"} Jul 02 22:09:21.601311 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:21.601275 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e0a26d6-f7c9-4789-bad6-a3798eb69013" path="/var/lib/kubelet/pods/8e0a26d6-f7c9-4789-bad6-a3798eb69013/volumes" Jul 02 22:09:28.131280 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:28.131242 2583 generic.go:356] "Generic (PLEG): container finished" podID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerID="bce578fe9e8dcf211b47c6026c90408288c6c181a9fe366aa9263707e014265e" exitCode=0 Jul 02 22:09:28.131713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:28.131326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerDied","Data":"bce578fe9e8dcf211b47c6026c90408288c6c181a9fe366aa9263707e014265e"} Jul 02 22:09:30.143174 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:30.143129 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerStarted","Data":"5b2e6ef73683d6d478ae82a250db13afa474760c137149011279a7a6755670a7"} Jul 02 22:09:31.149820 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:31.149784 2583 generic.go:356] "Generic (PLEG): container finished" podID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerID="5b2e6ef73683d6d478ae82a250db13afa474760c137149011279a7a6755670a7" exitCode=0 Jul 02 22:09:31.150171 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:31.149868 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerDied","Data":"5b2e6ef73683d6d478ae82a250db13afa474760c137149011279a7a6755670a7"} Jul 02 22:09:40.195743 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:40.195707 2583 generic.go:356] "Generic (PLEG): container finished" podID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerID="33fcb21cb744fdaf0bddd194f02a1301b94a7d5782f4ab176ad7e015ee0fab96" exitCode=0 Jul 02 22:09:40.196169 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:40.195760 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerDied","Data":"33fcb21cb744fdaf0bddd194f02a1301b94a7d5782f4ab176ad7e015ee0fab96"} Jul 02 22:09:41.379822 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.379765 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:41.442616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.442578 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle\") pod \"2f183026-5a10-48dd-a0c3-0d0a64337812\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " Jul 02 22:09:41.442820 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.442630 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkjgm\" (UniqueName: \"kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm\") pod \"2f183026-5a10-48dd-a0c3-0d0a64337812\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " Jul 02 22:09:41.442820 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.442676 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util\") pod \"2f183026-5a10-48dd-a0c3-0d0a64337812\" (UID: \"2f183026-5a10-48dd-a0c3-0d0a64337812\") " Jul 02 22:09:41.443434 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.443397 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle" (OuterVolumeSpecName: "bundle") pod "2f183026-5a10-48dd-a0c3-0d0a64337812" (UID: "2f183026-5a10-48dd-a0c3-0d0a64337812"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:41.444921 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.444891 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm" (OuterVolumeSpecName: "kube-api-access-kkjgm") pod "2f183026-5a10-48dd-a0c3-0d0a64337812" (UID: "2f183026-5a10-48dd-a0c3-0d0a64337812"). InnerVolumeSpecName "kube-api-access-kkjgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:41.447066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.447041 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util" (OuterVolumeSpecName: "util") pod "2f183026-5a10-48dd-a0c3-0d0a64337812" (UID: "2f183026-5a10-48dd-a0c3-0d0a64337812"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:41.544424 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.544337 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:41.544424 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.544369 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f183026-5a10-48dd-a0c3-0d0a64337812-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:41.544424 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:41.544379 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkjgm\" (UniqueName: \"kubernetes.io/projected/2f183026-5a10-48dd-a0c3-0d0a64337812-kube-api-access-kkjgm\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:42.206271 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:42.206231 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" event={"ID":"2f183026-5a10-48dd-a0c3-0d0a64337812","Type":"ContainerDied","Data":"c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc"} Jul 02 22:09:42.206445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:42.206281 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3aa2bc6152d53f50d33e85add355bf99b3d2aabea8997ef676aa484d44200cc" Jul 02 22:09:42.206445 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:42.206250 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tk8p7" Jul 02 22:09:46.029328 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:46.029275 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:47.554169 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:47.554130 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:48.027491 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027450 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds"] Jul 02 22:09:48.027849 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027832 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="util" Jul 02 22:09:48.027932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027852 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="util" Jul 02 22:09:48.027932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027887 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="extract" Jul 02 22:09:48.027932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027897 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="extract" Jul 02 22:09:48.027932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027914 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="pull" Jul 02 22:09:48.027932 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.027923 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="pull" Jul 02 22:09:48.028179 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.028001 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f183026-5a10-48dd-a0c3-0d0a64337812" containerName="extract" Jul 02 22:09:48.030156 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.030134 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.033020 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.033002 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:09:48.033127 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.033087 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-s96r2\"" Jul 02 22:09:48.033206 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.033193 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 22:09:48.045587 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.045556 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds"] Jul 02 22:09:48.106636 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.106592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc77a944-773b-41bd-8070-4b3d0ddcd885-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.106811 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.106738 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp6zv\" (UniqueName: \"kubernetes.io/projected/cc77a944-773b-41bd-8070-4b3d0ddcd885-kube-api-access-rp6zv\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.207177 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.207121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp6zv\" (UniqueName: \"kubernetes.io/projected/cc77a944-773b-41bd-8070-4b3d0ddcd885-kube-api-access-rp6zv\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.207408 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.207203 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc77a944-773b-41bd-8070-4b3d0ddcd885-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.207677 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.207652 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc77a944-773b-41bd-8070-4b3d0ddcd885-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.220063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.220040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp6zv\" (UniqueName: \"kubernetes.io/projected/cc77a944-773b-41bd-8070-4b3d0ddcd885-kube-api-access-rp6zv\") pod \"cert-manager-operator-controller-manager-55c444bf86-pfvds\" (UID: \"cc77a944-773b-41bd-8070-4b3d0ddcd885\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.340026 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.339932 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" Jul 02 22:09:48.481235 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:48.481204 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds"] Jul 02 22:09:48.485341 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:09:48.485285 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc77a944_773b_41bd_8070_4b3d0ddcd885.slice/crio-c65ba49f769c059be6b9ca1eefd977a48a8842c46d129b1ca78462d2e9e77dd7 WatchSource:0}: Error finding container c65ba49f769c059be6b9ca1eefd977a48a8842c46d129b1ca78462d2e9e77dd7: Status 404 returned error can't find the container with id c65ba49f769c059be6b9ca1eefd977a48a8842c46d129b1ca78462d2e9e77dd7 Jul 02 22:09:49.238621 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:49.238588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" event={"ID":"cc77a944-773b-41bd-8070-4b3d0ddcd885","Type":"ContainerStarted","Data":"c65ba49f769c059be6b9ca1eefd977a48a8842c46d129b1ca78462d2e9e77dd7"} Jul 02 22:09:49.607232 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:49.607138 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:49.667834 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:49.667801 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:51.251454 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:51.251411 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" event={"ID":"cc77a944-773b-41bd-8070-4b3d0ddcd885","Type":"ContainerStarted","Data":"9ca4b5885c6997da7ec179b7feac1281066161321351fdb83bb003893845deff"} Jul 02 22:09:51.275855 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:51.275782 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pfvds" podStartSLOduration=1.1349343219999999 podStartE2EDuration="3.275757393s" totalImagesPullingTime="2.140823071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:09:51.272610326 +0000 UTC m=+336.256822408" watchObservedRunningTime="2026-07-02 22:09:51.275757393 +0000 UTC m=+336.259969538" Jul 02 22:09:51.460615 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:51.460579 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:51.585050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:51.584961 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:51.747604 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:51.747567 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:52.324249 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.324214 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9"] Jul 02 22:09:52.326635 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.326616 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.329784 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.329760 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kwtd8\"" Jul 02 22:09:52.329784 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.329771 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 22:09:52.330687 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.330674 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 22:09:52.339853 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.339821 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9"] Jul 02 22:09:52.443178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.443140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.443178 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.443180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj66n\" (UniqueName: \"kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.443449 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.443204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.544637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.544599 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.544637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.544641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj66n\" (UniqueName: \"kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.544899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.544665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.544993 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.544974 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.545048 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.545009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.553139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.553103 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj66n\" (UniqueName: \"kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.636260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.636172 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:52.774588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:52.774560 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9"] Jul 02 22:09:52.776715 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:09:52.776671 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103169d6_d7d8_43d0_9401_52a918373919.slice/crio-04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9 WatchSource:0}: Error finding container 04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9: Status 404 returned error can't find the container with id 04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9 Jul 02 22:09:53.262393 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:53.262359 2583 generic.go:356] "Generic (PLEG): container finished" podID="103169d6-d7d8-43d0-9401-52a918373919" containerID="7d032a4f587967322f7e42bb10e6f3c73cdc77586a2ddc7855ad245a9e01fef6" exitCode=0 Jul 02 22:09:53.262561 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:53.262423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" event={"ID":"103169d6-d7d8-43d0-9401-52a918373919","Type":"ContainerDied","Data":"7d032a4f587967322f7e42bb10e6f3c73cdc77586a2ddc7855ad245a9e01fef6"} Jul 02 22:09:53.262561 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:53.262465 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" event={"ID":"103169d6-d7d8-43d0-9401-52a918373919","Type":"ContainerStarted","Data":"04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9"} Jul 02 22:09:53.842485 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:53.842387 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:54.084412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.084370 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:54.147415 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.147337 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms"] Jul 02 22:09:54.149626 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.149605 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.152463 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.152437 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 22:09:54.152669 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.152619 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 22:09:54.152791 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.152684 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-n6wnx\"" Jul 02 22:09:54.162036 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.162008 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms"] Jul 02 22:09:54.261001 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.260960 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.261220 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.261030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8prrj\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-kube-api-access-8prrj\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.362317 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.362272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.362489 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.362351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8prrj\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-kube-api-access-8prrj\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.372021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.371957 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.372187 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.372104 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8prrj\" (UniqueName: \"kubernetes.io/projected/609dec92-0c7c-422c-bfd3-acaa90c0e846-kube-api-access-8prrj\") pod \"cert-manager-cainjector-559cdd8c87-wc8ms\" (UID: \"609dec92-0c7c-422c-bfd3-acaa90c0e846\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.468734 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.468697 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" Jul 02 22:09:54.624770 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:54.624742 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms"] Jul 02 22:09:54.628010 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:09:54.627976 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod609dec92_0c7c_422c_bfd3_acaa90c0e846.slice/crio-9d48f19c5e10f74ec9c95fc7b2cf3e10444c8711c8153c623ba38e8b9bb515a4 WatchSource:0}: Error finding container 9d48f19c5e10f74ec9c95fc7b2cf3e10444c8711c8153c623ba38e8b9bb515a4: Status 404 returned error can't find the container with id 9d48f19c5e10f74ec9c95fc7b2cf3e10444c8711c8153c623ba38e8b9bb515a4 Jul 02 22:09:55.274941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:55.274903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" event={"ID":"609dec92-0c7c-422c-bfd3-acaa90c0e846","Type":"ContainerStarted","Data":"9d48f19c5e10f74ec9c95fc7b2cf3e10444c8711c8153c623ba38e8b9bb515a4"} Jul 02 22:09:56.281889 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:56.281853 2583 generic.go:356] "Generic (PLEG): container finished" podID="103169d6-d7d8-43d0-9401-52a918373919" containerID="19e8a5125f39002e96d305a165826472553e2c98a770e55cc1c677308ad4c978" exitCode=0 Jul 02 22:09:56.282276 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:56.281942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" event={"ID":"103169d6-d7d8-43d0-9401-52a918373919","Type":"ContainerDied","Data":"19e8a5125f39002e96d305a165826472553e2c98a770e55cc1c677308ad4c978"} Jul 02 22:09:56.411060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:56.411034 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:56.474726 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:56.474677 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:57.289313 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:57.289259 2583 generic.go:356] "Generic (PLEG): container finished" podID="103169d6-d7d8-43d0-9401-52a918373919" containerID="26d7e648501afbe0d6e8a8983e67ca2dc53fdf38e6ecebd43e96fabeec1e59ef" exitCode=0 Jul 02 22:09:57.289695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:57.289352 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" event={"ID":"103169d6-d7d8-43d0-9401-52a918373919","Type":"ContainerDied","Data":"26d7e648501afbe0d6e8a8983e67ca2dc53fdf38e6ecebd43e96fabeec1e59ef"} Jul 02 22:09:58.433344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.433318 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:58.497861 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.497823 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util\") pod \"103169d6-d7d8-43d0-9401-52a918373919\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " Jul 02 22:09:58.498036 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.497873 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle\") pod \"103169d6-d7d8-43d0-9401-52a918373919\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " Jul 02 22:09:58.498036 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.497903 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj66n\" (UniqueName: \"kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n\") pod \"103169d6-d7d8-43d0-9401-52a918373919\" (UID: \"103169d6-d7d8-43d0-9401-52a918373919\") " Jul 02 22:09:58.498273 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.498244 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle" (OuterVolumeSpecName: "bundle") pod "103169d6-d7d8-43d0-9401-52a918373919" (UID: "103169d6-d7d8-43d0-9401-52a918373919"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:58.500165 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.500139 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n" (OuterVolumeSpecName: "kube-api-access-jj66n") pod "103169d6-d7d8-43d0-9401-52a918373919" (UID: "103169d6-d7d8-43d0-9401-52a918373919"). InnerVolumeSpecName "kube-api-access-jj66n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:58.502087 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.502067 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util" (OuterVolumeSpecName: "util") pod "103169d6-d7d8-43d0-9401-52a918373919" (UID: "103169d6-d7d8-43d0-9401-52a918373919"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:58.599060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.598977 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj66n\" (UniqueName: \"kubernetes.io/projected/103169d6-d7d8-43d0-9401-52a918373919-kube-api-access-jj66n\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:58.599060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.599006 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:58.599060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:58.599016 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103169d6-d7d8-43d0-9401-52a918373919-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:09:59.300037 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:59.300001 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" Jul 02 22:09:59.300037 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:59.300022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq45t9" event={"ID":"103169d6-d7d8-43d0-9401-52a918373919","Type":"ContainerDied","Data":"04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9"} Jul 02 22:09:59.300260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:09:59.300072 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04f75684f9908aa7c8c5725cc637b9cc0dabf9240e0c8190d345ec317449d8e9" Jul 02 22:10:01.653895 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:01.653850 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:02.753336 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753286 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2fcp2"] Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753682 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="pull" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753696 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="pull" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753715 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="util" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753721 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="util" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753743 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="extract" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753753 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="extract" Jul 02 22:10:02.753845 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.753824 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="103169d6-d7d8-43d0-9401-52a918373919" containerName="extract" Jul 02 22:10:02.757260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.757239 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.760227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.760198 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dq8xw\"" Jul 02 22:10:02.768245 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.768211 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2fcp2"] Jul 02 22:10:02.837838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.837798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-bound-sa-token\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.837838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.837843 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kvhf\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-kube-api-access-5kvhf\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.939444 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.939405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-bound-sa-token\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.939444 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.939451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kvhf\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-kube-api-access-5kvhf\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.949146 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.949113 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-bound-sa-token\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:02.949295 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:02.949124 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kvhf\" (UniqueName: \"kubernetes.io/projected/96697c2f-d33e-4b22-8b80-f3c6b0bbf84e-kube-api-access-5kvhf\") pod \"cert-manager-55b4d954d5-2fcp2\" (UID: \"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e\") " pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:03.068152 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:03.068047 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2fcp2" Jul 02 22:10:03.594798 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:03.594774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2fcp2"] Jul 02 22:10:03.597279 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:03.597249 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96697c2f_d33e_4b22_8b80_f3c6b0bbf84e.slice/crio-77a48a52e3a40a4ba72deaa83f4e5c9ffe004c66d25b240cafb05526e35c3b7b WatchSource:0}: Error finding container 77a48a52e3a40a4ba72deaa83f4e5c9ffe004c66d25b240cafb05526e35c3b7b: Status 404 returned error can't find the container with id 77a48a52e3a40a4ba72deaa83f4e5c9ffe004c66d25b240cafb05526e35c3b7b Jul 02 22:10:04.317507 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.317468 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:04.327818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.327781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" event={"ID":"609dec92-0c7c-422c-bfd3-acaa90c0e846","Type":"ContainerStarted","Data":"2cb46fdfb0f80052c266b2f47742ed79092eac96ff709f63b27f95794bffa01f"} Jul 02 22:10:04.329350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.329321 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2fcp2" event={"ID":"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e","Type":"ContainerStarted","Data":"983031b927855cee411c7f34348c40aa5ee4d2e203dbce384972266a91e6c82c"} Jul 02 22:10:04.329482 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.329361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2fcp2" event={"ID":"96697c2f-d33e-4b22-8b80-f3c6b0bbf84e","Type":"ContainerStarted","Data":"77a48a52e3a40a4ba72deaa83f4e5c9ffe004c66d25b240cafb05526e35c3b7b"} Jul 02 22:10:04.349018 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.348954 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wc8ms" podStartSLOduration=1.461135134 podStartE2EDuration="10.348937039s" totalImagesPullingTime="8.887801905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:04.348887375 +0000 UTC m=+349.333099467" watchObservedRunningTime="2026-07-02 22:10:04.348937039 +0000 UTC m=+349.333149121" Jul 02 22:10:04.368355 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.368271 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2fcp2" podStartSLOduration=2.3682528019999998 podStartE2EDuration="2.368252802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:04.366629794 +0000 UTC m=+349.350841877" watchObservedRunningTime="2026-07-02 22:10:04.368252802 +0000 UTC m=+349.352464882" Jul 02 22:10:04.417415 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.417376 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:04.601021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.600931 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:04.700401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:04.700358 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:06.170046 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.170013 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8"] Jul 02 22:10:06.213398 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.213354 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8"] Jul 02 22:10:06.213590 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.213408 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.216377 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.216339 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 22:10:06.216528 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.216436 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 22:10:06.217652 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.217625 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bqljq\"" Jul 02 22:10:06.373840 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.373799 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf4vt\" (UniqueName: \"kubernetes.io/projected/e363df1e-eade-44e0-b06f-d9960f16547e-kube-api-access-hf4vt\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.374020 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.373878 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e363df1e-eade-44e0-b06f-d9960f16547e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.475287 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.475244 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e363df1e-eade-44e0-b06f-d9960f16547e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.475500 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.475360 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf4vt\" (UniqueName: \"kubernetes.io/projected/e363df1e-eade-44e0-b06f-d9960f16547e-kube-api-access-hf4vt\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.475641 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.475621 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e363df1e-eade-44e0-b06f-d9960f16547e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.484166 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.484134 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf4vt\" (UniqueName: \"kubernetes.io/projected/e363df1e-eade-44e0-b06f-d9960f16547e-kube-api-access-hf4vt\") pod \"openshift-lws-operator-fd8ccff4c-927t8\" (UID: \"e363df1e-eade-44e0-b06f-d9960f16547e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.522849 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.522812 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" Jul 02 22:10:06.692033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:06.692006 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8"] Jul 02 22:10:06.694796 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:06.694771 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode363df1e_eade_44e0_b06f_d9960f16547e.slice/crio-d3cccb5c4f9f5252c6ab596a4d04140bd82cb4b7002384b964a101e847863d1f WatchSource:0}: Error finding container d3cccb5c4f9f5252c6ab596a4d04140bd82cb4b7002384b964a101e847863d1f: Status 404 returned error can't find the container with id d3cccb5c4f9f5252c6ab596a4d04140bd82cb4b7002384b964a101e847863d1f Jul 02 22:10:07.343653 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:07.343569 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" event={"ID":"e363df1e-eade-44e0-b06f-d9960f16547e","Type":"ContainerStarted","Data":"d3cccb5c4f9f5252c6ab596a4d04140bd82cb4b7002384b964a101e847863d1f"} Jul 02 22:10:09.357577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:09.357545 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" event={"ID":"e363df1e-eade-44e0-b06f-d9960f16547e","Type":"ContainerStarted","Data":"9b923d892b2da34f83e802f4788ad19e9902023df93968fa5f09af9c4ab5359f"} Jul 02 22:10:09.377157 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:09.377093 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927t8" podStartSLOduration=1.1635029000000001 podStartE2EDuration="3.377069548s" totalImagesPullingTime="2.213566648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:09.375934404 +0000 UTC m=+354.360146484" watchObservedRunningTime="2026-07-02 22:10:09.377069548 +0000 UTC m=+354.361281634" Jul 02 22:10:10.099829 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:10.099793 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:10.200539 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:10.200501 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.741422 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.741381 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv"] Jul 02 22:10:11.744238 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.744220 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:11.747145 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.747121 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 22:10:11.747286 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.747120 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 22:10:11.747286 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.747207 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kwtd8\"" Jul 02 22:10:11.754897 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.754870 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv"] Jul 02 22:10:11.924474 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.924434 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7jt\" (UniqueName: \"kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:11.924694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.924496 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:11.924694 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.924576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:11.968809 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:11.968766 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:12.025374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.025264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7jt\" (UniqueName: \"kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.025374 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.025368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.025575 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.025422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.025827 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.025801 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.025865 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.025809 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.034296 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.034256 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7jt\" (UniqueName: \"kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.054081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.054043 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:12.191240 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.191212 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv"] Jul 02 22:10:12.193127 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:12.193085 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f242e9e_c6c9_4fb9_8872_99898805d40b.slice/crio-6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52 WatchSource:0}: Error finding container 6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52: Status 404 returned error can't find the container with id 6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52 Jul 02 22:10:12.373604 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.373567 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerID="d8ba868e06841e401dd77ad2cfaa5dfada8ba200f857737b8c63613b938e2694" exitCode=0 Jul 02 22:10:12.373793 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.373653 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" event={"ID":"4f242e9e-c6c9-4fb9-8872-99898805d40b","Type":"ContainerDied","Data":"d8ba868e06841e401dd77ad2cfaa5dfada8ba200f857737b8c63613b938e2694"} Jul 02 22:10:12.373793 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:12.373754 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" event={"ID":"4f242e9e-c6c9-4fb9-8872-99898805d40b","Type":"ContainerStarted","Data":"6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52"} Jul 02 22:10:13.381622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:13.381523 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerID="5d49be1c2c445bda499f76810353a05a84c66bd7efa58aa5dc472615bb905cc1" exitCode=0 Jul 02 22:10:13.381622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:13.381577 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" event={"ID":"4f242e9e-c6c9-4fb9-8872-99898805d40b","Type":"ContainerDied","Data":"5d49be1c2c445bda499f76810353a05a84c66bd7efa58aa5dc472615bb905cc1"} Jul 02 22:10:14.388895 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:14.388855 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerID="2d9f22078c680ef3dbf833dd92a18cdcde4b2dd3b45895cb730e4e56d090e117" exitCode=0 Jul 02 22:10:14.389329 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:14.388942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" event={"ID":"4f242e9e-c6c9-4fb9-8872-99898805d40b","Type":"ContainerDied","Data":"2d9f22078c680ef3dbf833dd92a18cdcde4b2dd3b45895cb730e4e56d090e117"} Jul 02 22:10:15.531460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.531432 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:15.659219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.659112 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util\") pod \"4f242e9e-c6c9-4fb9-8872-99898805d40b\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " Jul 02 22:10:15.659219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.659181 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle\") pod \"4f242e9e-c6c9-4fb9-8872-99898805d40b\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " Jul 02 22:10:15.659219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.659219 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx7jt\" (UniqueName: \"kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt\") pod \"4f242e9e-c6c9-4fb9-8872-99898805d40b\" (UID: \"4f242e9e-c6c9-4fb9-8872-99898805d40b\") " Jul 02 22:10:15.659922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.659891 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle" (OuterVolumeSpecName: "bundle") pod "4f242e9e-c6c9-4fb9-8872-99898805d40b" (UID: "4f242e9e-c6c9-4fb9-8872-99898805d40b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:15.661401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.661372 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt" (OuterVolumeSpecName: "kube-api-access-jx7jt") pod "4f242e9e-c6c9-4fb9-8872-99898805d40b" (UID: "4f242e9e-c6c9-4fb9-8872-99898805d40b"). InnerVolumeSpecName "kube-api-access-jx7jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:15.664655 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.664630 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util" (OuterVolumeSpecName: "util") pod "4f242e9e-c6c9-4fb9-8872-99898805d40b" (UID: "4f242e9e-c6c9-4fb9-8872-99898805d40b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:15.760167 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.760129 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:10:15.760167 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.760159 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f242e9e-c6c9-4fb9-8872-99898805d40b-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:10:15.760167 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:15.760169 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx7jt\" (UniqueName: \"kubernetes.io/projected/4f242e9e-c6c9-4fb9-8872-99898805d40b-kube-api-access-jx7jt\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:10:16.402021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:16.401987 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" Jul 02 22:10:16.402021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:16.402016 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdx8gv" event={"ID":"4f242e9e-c6c9-4fb9-8872-99898805d40b","Type":"ContainerDied","Data":"6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52"} Jul 02 22:10:16.402237 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:16.402060 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6456a0dbb5990a05528a1871d73aa6bc3a5ac173c1d345898d9d0acad3f56c52" Jul 02 22:10:25.451879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:25.451842 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:25.557859 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:25.557827 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.054044 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.054010 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.309297 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.309190 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.415001 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.414963 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.484210 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484177 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx"] Jul 02 22:10:26.484606 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484592 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="util" Jul 02 22:10:26.484606 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484607 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="util" Jul 02 22:10:26.484689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484623 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="pull" Jul 02 22:10:26.484689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484628 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="pull" Jul 02 22:10:26.484689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484643 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="extract" Jul 02 22:10:26.484689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484651 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="extract" Jul 02 22:10:26.484812 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.484717 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f242e9e-c6c9-4fb9-8872-99898805d40b" containerName="extract" Jul 02 22:10:26.487185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.487165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.489839 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.489817 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 22:10:26.490493 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.490476 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 22:10:26.490616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.490476 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 22:10:26.490686 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.490636 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 22:10:26.490767 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.490749 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-268h4\"" Jul 02 22:10:26.514090 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.514053 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx"] Jul 02 22:10:26.549769 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.549734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-webhook-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.549941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.549789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.549941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.549857 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-manifests\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.549941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.549899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-tmp\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.550086 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.549957 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4xwr\" (UniqueName: \"kubernetes.io/projected/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-kube-api-access-c4xwr\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.649833 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.649740 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.650729 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.650705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.650792 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.650745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-manifests\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.650792 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.650767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-tmp\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.650898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.650812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4xwr\" (UniqueName: \"kubernetes.io/projected/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-kube-api-access-c4xwr\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.650898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.650842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-webhook-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.651151 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.651128 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-manifests\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.653216 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.653189 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-tmp\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.653412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.653393 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.653554 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.653533 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-webhook-cert\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.660408 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.660381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4xwr\" (UniqueName: \"kubernetes.io/projected/3fef31d2-ca5c-43ac-bfdf-6f59b569c040-kube-api-access-c4xwr\") pod \"opendatahub-operator-controller-manager-b66c9c9d4-vrtcx\" (UID: \"3fef31d2-ca5c-43ac-bfdf-6f59b569c040\") " pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.745778 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.745732 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.799321 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.799276 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:26.863758 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.863722 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:26.949909 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.949857 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx"] Jul 02 22:10:26.952318 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:26.952275 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fef31d2_ca5c_43ac_bfdf_6f59b569c040.slice/crio-5d4437a1008e22b708dbdf5a287f7918d69a7600ae283730f815d1c4325f8a12 WatchSource:0}: Error finding container 5d4437a1008e22b708dbdf5a287f7918d69a7600ae283730f815d1c4325f8a12: Status 404 returned error can't find the container with id 5d4437a1008e22b708dbdf5a287f7918d69a7600ae283730f815d1c4325f8a12 Jul 02 22:10:26.959501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:26.959468 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:27.059913 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:27.059879 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:27.169204 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:27.169152 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:27.263383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:27.263272 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:27.456713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:27.456672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" event={"ID":"3fef31d2-ca5c-43ac-bfdf-6f59b569c040","Type":"ContainerStarted","Data":"5d4437a1008e22b708dbdf5a287f7918d69a7600ae283730f815d1c4325f8a12"} Jul 02 22:10:29.465359 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:29.465281 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:29.567109 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:29.567067 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:30.476030 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:30.475995 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" event={"ID":"3fef31d2-ca5c-43ac-bfdf-6f59b569c040","Type":"ContainerStarted","Data":"e7000107aed372bd9f5d6d99305aef5122d5f43172d0dcd44f5b3fe1c44f76ac"} Jul 02 22:10:31.482537 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:31.482503 2583 generic.go:356] "Generic (PLEG): container finished" podID="3fef31d2-ca5c-43ac-bfdf-6f59b569c040" containerID="e7000107aed372bd9f5d6d99305aef5122d5f43172d0dcd44f5b3fe1c44f76ac" exitCode=0 Jul 02 22:10:31.482930 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:31.482561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" event={"ID":"3fef31d2-ca5c-43ac-bfdf-6f59b569c040","Type":"ContainerDied","Data":"e7000107aed372bd9f5d6d99305aef5122d5f43172d0dcd44f5b3fe1c44f76ac"} Jul 02 22:10:32.489194 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:32.489159 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" event={"ID":"3fef31d2-ca5c-43ac-bfdf-6f59b569c040","Type":"ContainerStarted","Data":"22c7009f7ce990ff2c7592d7eed48730aadaddc129bc96f5ffe12a3f54480006"} Jul 02 22:10:32.489574 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:32.489288 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:32.514907 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:32.514851 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" podStartSLOduration=2.042941666 podStartE2EDuration="6.514832252s" totalImagesPullingTime="3.35678447s" totalInitContainerRuntime="1.115106116s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:32.512822572 +0000 UTC m=+377.497034650" watchObservedRunningTime="2026-07-02 22:10:32.514832252 +0000 UTC m=+377.499044333" Jul 02 22:10:34.215346 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:34.215283 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:34.319912 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:34.319877 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:38.045682 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:38.045643 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:38.145780 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:38.145744 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:42.431598 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:42.431565 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:42.897814 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:42.897716 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:43.031765 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:43.031728 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:43.497148 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:43.497117 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b66c9c9d4-vrtcx" Jul 02 22:10:45.445881 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:45.445844 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:45.617646 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:45.617599 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:46.132550 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.132490 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:46.198263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.198218 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl"] Jul 02 22:10:46.201148 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.201107 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.204597 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.204575 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 22:10:46.205407 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.205380 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 22:10:46.224885 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.224851 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl"] Jul 02 22:10:46.280577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.280544 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5ws9s"] Jul 02 22:10:46.282783 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.282766 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.285616 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.285592 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8s6d5\"" Jul 02 22:10:46.285749 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.285716 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 22:10:46.299893 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.299861 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5ws9s"] Jul 02 22:10:46.323065 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.323026 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44aded33-eedb-4d47-aa97-780abfb2e300-tmp\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.323259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.323154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44aded33-eedb-4d47-aa97-780abfb2e300-tls-certs\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.323339 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.323259 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz9z8\" (UniqueName: \"kubernetes.io/projected/44aded33-eedb-4d47-aa97-780abfb2e300-kube-api-access-rz9z8\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.331872 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.331842 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:46.423073 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.422973 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:46.424050 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.424031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz9z8\" (UniqueName: \"kubernetes.io/projected/44aded33-eedb-4d47-aa97-780abfb2e300-kube-api-access-rz9z8\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.424118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.424063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw8gk\" (UniqueName: \"kubernetes.io/projected/92593e17-87ff-41b4-8577-8d94d0c6c608-kube-api-access-dw8gk\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.424118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.424102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44aded33-eedb-4d47-aa97-780abfb2e300-tmp\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.424197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.424138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.424266 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.424249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44aded33-eedb-4d47-aa97-780abfb2e300-tls-certs\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.426512 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.426491 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44aded33-eedb-4d47-aa97-780abfb2e300-tmp\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.426730 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.426711 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44aded33-eedb-4d47-aa97-780abfb2e300-tls-certs\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.439424 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.439395 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz9z8\" (UniqueName: \"kubernetes.io/projected/44aded33-eedb-4d47-aa97-780abfb2e300-kube-api-access-rz9z8\") pod \"kube-auth-proxy-6f57454dd4-zk8tl\" (UID: \"44aded33-eedb-4d47-aa97-780abfb2e300\") " pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.512032 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.511992 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" Jul 02 22:10:46.524875 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.524809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8gk\" (UniqueName: \"kubernetes.io/projected/92593e17-87ff-41b4-8577-8d94d0c6c608-kube-api-access-dw8gk\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.524875 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.524861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.525081 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.524883 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:46.525081 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:46.525022 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 22:10:46.525715 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:46.525699 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert podName:92593e17-87ff-41b4-8577-8d94d0c6c608 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:47.025667915 +0000 UTC m=+392.009879989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert") pod "odh-model-controller-7998c67dc4-5ws9s" (UID: "92593e17-87ff-41b4-8577-8d94d0c6c608") : secret "odh-model-controller-webhook-cert" not found Jul 02 22:10:46.536708 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.536681 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8gk\" (UniqueName: \"kubernetes.io/projected/92593e17-87ff-41b4-8577-8d94d0c6c608-kube-api-access-dw8gk\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:46.681526 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.681489 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl"] Jul 02 22:10:46.684040 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:46.684013 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44aded33_eedb_4d47_aa97_780abfb2e300.slice/crio-fe9e789179e5a01b75d8692e1b35ecffbea11fd7924e6a63d3efbd196c7426c3 WatchSource:0}: Error finding container fe9e789179e5a01b75d8692e1b35ecffbea11fd7924e6a63d3efbd196c7426c3: Status 404 returned error can't find the container with id fe9e789179e5a01b75d8692e1b35ecffbea11fd7924e6a63d3efbd196c7426c3 Jul 02 22:10:46.934952 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:46.934919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:47.028884 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.028849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:47.031390 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.031360 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92593e17-87ff-41b4-8577-8d94d0c6c608-cert\") pod \"odh-model-controller-7998c67dc4-5ws9s\" (UID: \"92593e17-87ff-41b4-8577-8d94d0c6c608\") " pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:47.134999 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.134961 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4qrvg"] Jul 02 22:10:47.137688 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.137666 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.145138 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.145037 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:47.147572 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.147550 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-l4bp9\"" Jul 02 22:10:47.148103 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.148080 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 22:10:47.194031 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.193935 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:47.194248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.194227 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4qrvg"] Jul 02 22:10:47.231446 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.231406 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.231637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.231459 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqxc\" (UniqueName: \"kubernetes.io/projected/ff5bb769-5627-4a49-ad70-5f368cfd9438-kube-api-access-4fqxc\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.332095 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.332058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.332253 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.332115 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqxc\" (UniqueName: \"kubernetes.io/projected/ff5bb769-5627-4a49-ad70-5f368cfd9438-kube-api-access-4fqxc\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.332253 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:47.332206 2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 02 22:10:47.332393 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:47.332276 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs podName:ff5bb769-5627-4a49-ad70-5f368cfd9438 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:47.832257912 +0000 UTC m=+392.816469971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs") pod "model-serving-api-54d89bd5fc-4qrvg" (UID: "ff5bb769-5627-4a49-ad70-5f368cfd9438") : secret "model-serving-api-tls" not found Jul 02 22:10:47.349012 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.348974 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqxc\" (UniqueName: \"kubernetes.io/projected/ff5bb769-5627-4a49-ad70-5f368cfd9438-kube-api-access-4fqxc\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.367462 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.367435 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5ws9s"] Jul 02 22:10:47.369961 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:47.369939 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92593e17_87ff_41b4_8577_8d94d0c6c608.slice/crio-50ebbcd7805b2f0067ee5514f4392c3d4fe34a43d8336ae828aaf2b9e37fd294 WatchSource:0}: Error finding container 50ebbcd7805b2f0067ee5514f4392c3d4fe34a43d8336ae828aaf2b9e37fd294: Status 404 returned error can't find the container with id 50ebbcd7805b2f0067ee5514f4392c3d4fe34a43d8336ae828aaf2b9e37fd294 Jul 02 22:10:47.522100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.522008 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:47.576007 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.575961 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" event={"ID":"44aded33-eedb-4d47-aa97-780abfb2e300","Type":"ContainerStarted","Data":"fe9e789179e5a01b75d8692e1b35ecffbea11fd7924e6a63d3efbd196c7426c3"} Jul 02 22:10:47.577852 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.577815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" event={"ID":"92593e17-87ff-41b4-8577-8d94d0c6c608","Type":"ContainerStarted","Data":"50ebbcd7805b2f0067ee5514f4392c3d4fe34a43d8336ae828aaf2b9e37fd294"} Jul 02 22:10:47.837002 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.836911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:47.839730 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:47.839704 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5bb769-5627-4a49-ad70-5f368cfd9438-tls-certs\") pod \"model-serving-api-54d89bd5fc-4qrvg\" (UID: \"ff5bb769-5627-4a49-ad70-5f368cfd9438\") " pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:48.048670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.048633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:48.237951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.237736 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4qrvg"] Jul 02 22:10:48.241608 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:48.241556 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5bb769_5627_4a49_ad70_5f368cfd9438.slice/crio-243153a7848439f7f39c9fcb315ec9b747f8b773a29e17c1596809af455903e2 WatchSource:0}: Error finding container 243153a7848439f7f39c9fcb315ec9b747f8b773a29e17c1596809af455903e2: Status 404 returned error can't find the container with id 243153a7848439f7f39c9fcb315ec9b747f8b773a29e17c1596809af455903e2 Jul 02 22:10:48.262286 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.262234 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:10:48.265465 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.265373 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:48.265465 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.265444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.268210 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.268184 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 22:10:48.268369 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.268189 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 22:10:48.268560 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.268511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gvqvf\"" Jul 02 22:10:48.268787 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.268770 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 22:10:48.268863 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.268818 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 22:10:48.286537 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.286510 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:10:48.343061 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343022 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjwx6\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343493 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343493 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343422 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.343601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.343500 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.362491 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.362400 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444229 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444288 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjwx6\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.444898 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.444534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.445527 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.445496 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.447785 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.447702 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.447924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.447797 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.448425 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.448384 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.449371 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.449296 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.454712 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.454668 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.454825 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.454738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjwx6\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6\") pod \"istiod-openshift-gateway-7574459656-tdrkw\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.579038 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.578765 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:48.588389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.588322 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" event={"ID":"ff5bb769-5627-4a49-ad70-5f368cfd9438","Type":"ContainerStarted","Data":"243153a7848439f7f39c9fcb315ec9b747f8b773a29e17c1596809af455903e2"} Jul 02 22:10:48.769638 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:48.768439 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:10:48.776333 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:48.775834 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a38a7c9_be3b_40c4_9f15_c6e772d11b49.slice/crio-207a1807d6bb299db9e2ea3682f7e5ec001e84bfb14fb7bca6a2f6d5abe9bc8e WatchSource:0}: Error finding container 207a1807d6bb299db9e2ea3682f7e5ec001e84bfb14fb7bca6a2f6d5abe9bc8e: Status 404 returned error can't find the container with id 207a1807d6bb299db9e2ea3682f7e5ec001e84bfb14fb7bca6a2f6d5abe9bc8e Jul 02 22:10:49.541826 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:49.541784 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:49.602963 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:49.602919 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" event={"ID":"7a38a7c9-be3b-40c4-9f15-c6e772d11b49","Type":"ContainerStarted","Data":"207a1807d6bb299db9e2ea3682f7e5ec001e84bfb14fb7bca6a2f6d5abe9bc8e"} Jul 02 22:10:52.543524 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:52.543487 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 02 22:10:52.543814 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:52.543558 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 02 22:10:53.502033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.501986 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:53.592766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.592726 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:53.629519 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.629479 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" event={"ID":"92593e17-87ff-41b4-8577-8d94d0c6c608","Type":"ContainerStarted","Data":"0d5feb24df45442509e1e2a4d2ffc3ade24e1faba22803d59b2bace4b3bf65a2"} Jul 02 22:10:53.629697 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.629637 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:10:53.631075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.631051 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" event={"ID":"7a38a7c9-be3b-40c4-9f15-c6e772d11b49","Type":"ContainerStarted","Data":"889952a50d477c68776fc099c9ee2d3848c8ab045c4bcfb80d724ef842658c29"} Jul 02 22:10:53.631226 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.631162 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:53.632735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.632706 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" event={"ID":"ff5bb769-5627-4a49-ad70-5f368cfd9438","Type":"ContainerStarted","Data":"32279e46ff0f220767e1dc24b821231e912284da8254b0b90eed759b1a817783"} Jul 02 22:10:53.632873 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.632816 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:10:53.633223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.633194 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-tdrkw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 22:10:53.633335 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.633245 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 22:10:53.634344 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.634260 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" event={"ID":"44aded33-eedb-4d47-aa97-780abfb2e300","Type":"ContainerStarted","Data":"20fe61f1a1664a397ad06406c6c9186df16d6355db1002ea2ba3015186ca798c"} Jul 02 22:10:53.657902 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.657849 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" podStartSLOduration=2.493736429 podStartE2EDuration="7.657832033s" totalImagesPullingTime="5.164095604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:53.656345401 +0000 UTC m=+398.640557486" watchObservedRunningTime="2026-07-02 22:10:53.657832033 +0000 UTC m=+398.642044114" Jul 02 22:10:53.711768 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.711713 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" podStartSLOduration=1.9489072200000002 podStartE2EDuration="5.711694188s" totalImagesPullingTime="3.762786968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:53.709185732 +0000 UTC m=+398.693397813" watchObservedRunningTime="2026-07-02 22:10:53.711694188 +0000 UTC m=+398.695906270" Jul 02 22:10:53.758925 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.758801 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f57454dd4-zk8tl" podStartSLOduration=2.8007258779999997 podStartE2EDuration="7.758776323s" totalImagesPullingTime="4.958050445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:53.756270006 +0000 UTC m=+398.740482088" watchObservedRunningTime="2026-07-02 22:10:53.758776323 +0000 UTC m=+398.742988405" Jul 02 22:10:53.799664 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:53.799606 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" podStartSLOduration=2.539462456 podStartE2EDuration="6.799586566s" totalImagesPullingTime="4.26012411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:53.7968204 +0000 UTC m=+398.781032483" watchObservedRunningTime="2026-07-02 22:10:53.799586566 +0000 UTC m=+398.783798647" Jul 02 22:10:54.475753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:54.475720 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:54.640949 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:54.640920 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:10:54.834669 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:54.834561 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:55.455269 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:55.455226 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:57.461478 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.461429 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:57.545336 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.545279 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n"] Jul 02 22:10:57.547967 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.547948 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.552061 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.552038 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 22:10:57.552760 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.552745 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dghr8\"" Jul 02 22:10:57.582473 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.582438 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n"] Jul 02 22:10:57.638924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.638874 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.639113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.638964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsng\" (UniqueName: \"kubernetes.io/projected/0c1bc535-d935-492f-bbe3-ab279cfd0356-kube-api-access-knsng\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.649279 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.649243 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5"] Jul 02 22:10:57.651822 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.651806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.654892 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.654857 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:57.655411 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.655396 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rmq5w\"" Jul 02 22:10:57.655891 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.655869 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 22:10:57.681664 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.681628 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5"] Jul 02 22:10:57.740083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.739977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsng\" (UniqueName: \"kubernetes.io/projected/0c1bc535-d935-492f-bbe3-ab279cfd0356-kube-api-access-knsng\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.740083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.740028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.740083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.740075 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5lwm\" (UniqueName: \"kubernetes.io/projected/34ac77d9-965e-4bf0-9dbd-7886dff6add5-kube-api-access-d5lwm\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.740475 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.740168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.740475 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:57.740287 2583 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 22:10:57.740475 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:57.740408 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert podName:0c1bc535-d935-492f-bbe3-ab279cfd0356 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:58.240390356 +0000 UTC m=+403.224602423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert") pod "kserve-controller-manager-d6f4b65b8-jkb7n" (UID: "0c1bc535-d935-492f-bbe3-ab279cfd0356") : secret "kserve-webhook-server-cert" not found Jul 02 22:10:57.749609 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.749581 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsng\" (UniqueName: \"kubernetes.io/projected/0c1bc535-d935-492f-bbe3-ab279cfd0356-kube-api-access-knsng\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:57.826218 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.826173 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:57.841248 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.841215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.841426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.841266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5lwm\" (UniqueName: \"kubernetes.io/projected/34ac77d9-965e-4bf0-9dbd-7886dff6add5-kube-api-access-d5lwm\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.841426 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:57.841362 2583 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 02 22:10:57.841426 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:10:57.841427 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert podName:34ac77d9-965e-4bf0-9dbd-7886dff6add5 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:58.341410327 +0000 UTC m=+403.325622386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert") pod "llmisvc-controller-manager-777c6bd7d5-r7bw5" (UID: "34ac77d9-965e-4bf0-9dbd-7886dff6add5") : secret "llmisvc-webhook-server-cert" not found Jul 02 22:10:57.852607 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.852575 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5lwm\" (UniqueName: \"kubernetes.io/projected/34ac77d9-965e-4bf0-9dbd-7886dff6add5-kube-api-access-d5lwm\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:57.919584 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:57.919546 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.239148 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.239115 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.246215 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.246184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:58.248651 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.248628 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c1bc535-d935-492f-bbe3-ab279cfd0356-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkb7n\" (UID: \"0c1bc535-d935-492f-bbe3-ab279cfd0356\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:58.347112 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.347069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:58.349563 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.349531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34ac77d9-965e-4bf0-9dbd-7886dff6add5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r7bw5\" (UID: \"34ac77d9-965e-4bf0-9dbd-7886dff6add5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:58.363515 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.363483 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.458569 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.458527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:10:58.562447 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.562404 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:10:58.614798 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.614555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n"] Jul 02 22:10:58.617575 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:58.617529 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1bc535_d935_492f_bbe3_ab279cfd0356.slice/crio-7be957fb13e9640c688735bdddb7be441837d9320b89a61596426a587af05bd0 WatchSource:0}: Error finding container 7be957fb13e9640c688735bdddb7be441837d9320b89a61596426a587af05bd0: Status 404 returned error can't find the container with id 7be957fb13e9640c688735bdddb7be441837d9320b89a61596426a587af05bd0 Jul 02 22:10:58.666737 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.666685 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" event={"ID":"0c1bc535-d935-492f-bbe3-ab279cfd0356","Type":"ContainerStarted","Data":"7be957fb13e9640c688735bdddb7be441837d9320b89a61596426a587af05bd0"} Jul 02 22:10:58.714782 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:58.714737 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5"] Jul 02 22:10:58.717086 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:10:58.717060 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34ac77d9_965e_4bf0_9dbd_7886dff6add5.slice/crio-3292df5d7d20c663340f16830c1b4f889fbbd3d7d6c5793b0b1a189d6d887b75 WatchSource:0}: Error finding container 3292df5d7d20c663340f16830c1b4f889fbbd3d7d6c5793b0b1a189d6d887b75: Status 404 returned error can't find the container with id 3292df5d7d20c663340f16830c1b4f889fbbd3d7d6c5793b0b1a189d6d887b75 Jul 02 22:10:59.673941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:10:59.673902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" event={"ID":"34ac77d9-965e-4bf0-9dbd-7886dff6add5","Type":"ContainerStarted","Data":"3292df5d7d20c663340f16830c1b4f889fbbd3d7d6c5793b0b1a189d6d887b75"} Jul 02 22:11:00.451463 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.451424 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:11:00.451730 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.451655 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" containerID="cri-o://889952a50d477c68776fc099c9ee2d3848c8ab045c4bcfb80d724ef842658c29" gracePeriod=30 Jul 02 22:11:00.684183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.684144 2583 generic.go:356] "Generic (PLEG): container finished" podID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerID="889952a50d477c68776fc099c9ee2d3848c8ab045c4bcfb80d724ef842658c29" exitCode=0 Jul 02 22:11:00.684628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.684233 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" event={"ID":"7a38a7c9-be3b-40c4-9f15-c6e772d11b49","Type":"ContainerDied","Data":"889952a50d477c68776fc099c9ee2d3848c8ab045c4bcfb80d724ef842658c29"} Jul 02 22:11:00.738935 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.738908 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:11:00.870685 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870648 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.870888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870722 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.870888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870767 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.870888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870794 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.870888 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870845 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.871076 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870904 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.871076 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.870938 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjwx6\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6\") pod \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\" (UID: \"7a38a7c9-be3b-40c4-9f15-c6e772d11b49\") " Jul 02 22:11:00.871389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.871355 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:11:00.874188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.874147 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token" (OuterVolumeSpecName: "istio-token") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:11:00.874383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.874350 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:11:00.874477 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.874387 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs" (OuterVolumeSpecName: "local-certs") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:11:00.874868 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.874811 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts" (OuterVolumeSpecName: "cacerts") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:11:00.875166 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.874926 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:11:00.875166 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.875135 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6" (OuterVolumeSpecName: "kube-api-access-gjwx6") pod "7a38a7c9-be3b-40c4-9f15-c6e772d11b49" (UID: "7a38a7c9-be3b-40c4-9f15-c6e772d11b49"). InnerVolumeSpecName "kube-api-access-gjwx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:11:00.972337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972228 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-token\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972266 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-local-certs\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972281 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-ca-configmap\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972323 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-kubeconfig\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972337 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972340 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-cacerts\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972660 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972353 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-istio-csr-dns-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:00.972660 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:00.972367 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjwx6\" (UniqueName: \"kubernetes.io/projected/7a38a7c9-be3b-40c4-9f15-c6e772d11b49-kube-api-access-gjwx6\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:01.636905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.636864 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:01.699383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.699339 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" event={"ID":"7a38a7c9-be3b-40c4-9f15-c6e772d11b49","Type":"ContainerDied","Data":"207a1807d6bb299db9e2ea3682f7e5ec001e84bfb14fb7bca6a2f6d5abe9bc8e"} Jul 02 22:11:01.699383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.699372 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" Jul 02 22:11:01.699891 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.699453 2583 scope.go:117] "RemoveContainer" containerID="889952a50d477c68776fc099c9ee2d3848c8ab045c4bcfb80d724ef842658c29" Jul 02 22:11:01.722824 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.722782 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:11:01.724330 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.724265 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:01.732816 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:01.732780 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw"] Jul 02 22:11:02.706668 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.706630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" event={"ID":"34ac77d9-965e-4bf0-9dbd-7886dff6add5","Type":"ContainerStarted","Data":"b8236b5e70dbc6df665a973c895a3557b570b1f888b5b6c0c439207e6ce9a03f"} Jul 02 22:11:02.707086 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.706750 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:11:02.709211 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.709181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" event={"ID":"0c1bc535-d935-492f-bbe3-ab279cfd0356","Type":"ContainerStarted","Data":"faf07a8c7938e5cb150c536c67ed0764ded5722132de63eac4236139e60fdb19"} Jul 02 22:11:02.709379 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.709340 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:11:02.779625 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.779570 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" podStartSLOduration=2.013794558 podStartE2EDuration="5.779553871s" totalImagesPullingTime="3.765759313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:11:02.777824907 +0000 UTC m=+407.762036987" watchObservedRunningTime="2026-07-02 22:11:02.779553871 +0000 UTC m=+407.763765952" Jul 02 22:11:02.780762 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:02.780729 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" podStartSLOduration=2.108374723 podStartE2EDuration="5.780719118s" totalImagesPullingTime="3.672344395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:11:02.746998133 +0000 UTC m=+407.731210216" watchObservedRunningTime="2026-07-02 22:11:02.780719118 +0000 UTC m=+407.764931199" Jul 02 22:11:03.601537 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:03.601491 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" path="/var/lib/kubelet/pods/7a38a7c9-be3b-40c4-9f15-c6e772d11b49/volumes" Jul 02 22:11:04.643032 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:04.643000 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5ws9s" Jul 02 22:11:04.644521 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:04.644501 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4qrvg" Jul 02 22:11:05.640709 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:05.640673 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-tdrkw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:11:05.640933 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:05.640731 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tdrkw" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.41:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:11:33.716744 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:33.716711 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r7bw5" Jul 02 22:11:33.719972 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:33.719943 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkb7n" Jul 02 22:11:46.886961 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:46.886866 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:46.987922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:46.987888 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:47.041869 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.041833 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:47.042210 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.042197 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" Jul 02 22:11:47.042260 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.042211 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" Jul 02 22:11:47.042327 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.042281 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a38a7c9-be3b-40c4-9f15-c6e772d11b49" containerName="discovery" Jul 02 22:11:47.045263 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.045246 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:47.048404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.048377 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vvwvr\"" Jul 02 22:11:47.048544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.048426 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 22:11:47.049527 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.049508 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 22:11:47.061506 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.061466 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:47.099547 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.099514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r856b\" (UniqueName: \"kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b\") pod \"kuadrant-operator-catalog-r6vfm\" (UID: \"df96f069-d19a-47e7-ad3d-6ee097336137\") " pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:47.119146 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.119111 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:47.200102 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.200070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r856b\" (UniqueName: \"kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b\") pod \"kuadrant-operator-catalog-r6vfm\" (UID: \"df96f069-d19a-47e7-ad3d-6ee097336137\") " pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:47.211140 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.211103 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r856b\" (UniqueName: \"kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b\") pod \"kuadrant-operator-catalog-r6vfm\" (UID: \"df96f069-d19a-47e7-ad3d-6ee097336137\") " pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:47.355412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.355374 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:47.392601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.392558 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:47.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.493559 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:47.495919 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:11:47.495887 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf96f069_d19a_47e7_ad3d_6ee097336137.slice/crio-8974efc45127804f566a3ff6f983093923cd5f027f4524cf7747c04af8dc0d9d WatchSource:0}: Error finding container 8974efc45127804f566a3ff6f983093923cd5f027f4524cf7747c04af8dc0d9d: Status 404 returned error can't find the container with id 8974efc45127804f566a3ff6f983093923cd5f027f4524cf7747c04af8dc0d9d Jul 02 22:11:47.613198 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.613158 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h87w4"] Jul 02 22:11:47.617656 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.617633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:47.638334 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.638280 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h87w4"] Jul 02 22:11:47.703144 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.703109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clg29\" (UniqueName: \"kubernetes.io/projected/73f347cf-0daf-407b-ad58-f7499b7d7b55-kube-api-access-clg29\") pod \"kuadrant-operator-catalog-h87w4\" (UID: \"73f347cf-0daf-407b-ad58-f7499b7d7b55\") " pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:47.804419 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.804329 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clg29\" (UniqueName: \"kubernetes.io/projected/73f347cf-0daf-407b-ad58-f7499b7d7b55-kube-api-access-clg29\") pod \"kuadrant-operator-catalog-h87w4\" (UID: \"73f347cf-0daf-407b-ad58-f7499b7d7b55\") " pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:47.814166 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.814133 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clg29\" (UniqueName: \"kubernetes.io/projected/73f347cf-0daf-407b-ad58-f7499b7d7b55-kube-api-access-clg29\") pod \"kuadrant-operator-catalog-h87w4\" (UID: \"73f347cf-0daf-407b-ad58-f7499b7d7b55\") " pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:47.927951 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.927910 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:47.943071 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:47.943031 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" event={"ID":"df96f069-d19a-47e7-ad3d-6ee097336137","Type":"ContainerStarted","Data":"8974efc45127804f566a3ff6f983093923cd5f027f4524cf7747c04af8dc0d9d"} Jul 02 22:11:48.071830 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:48.071803 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h87w4"] Jul 02 22:11:48.104465 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:11:48.104420 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73f347cf_0daf_407b_ad58_f7499b7d7b55.slice/crio-885d8e8f87e73d160bf5929ae118a24cb9ccbf299e48391f0f1dfae7af35d0d9 WatchSource:0}: Error finding container 885d8e8f87e73d160bf5929ae118a24cb9ccbf299e48391f0f1dfae7af35d0d9: Status 404 returned error can't find the container with id 885d8e8f87e73d160bf5929ae118a24cb9ccbf299e48391f0f1dfae7af35d0d9 Jul 02 22:11:48.954131 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:48.954063 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" event={"ID":"73f347cf-0daf-407b-ad58-f7499b7d7b55","Type":"ContainerStarted","Data":"885d8e8f87e73d160bf5929ae118a24cb9ccbf299e48391f0f1dfae7af35d0d9"} Jul 02 22:11:49.962386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:49.962343 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" event={"ID":"73f347cf-0daf-407b-ad58-f7499b7d7b55","Type":"ContainerStarted","Data":"30ce11ecb1af74db0ec59f1cc84b2b5c70ef414c48204367f5924747a3656bc5"} Jul 02 22:11:49.964100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:49.964068 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" event={"ID":"df96f069-d19a-47e7-ad3d-6ee097336137","Type":"ContainerStarted","Data":"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0"} Jul 02 22:11:49.964236 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:49.964073 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" podUID="df96f069-d19a-47e7-ad3d-6ee097336137" containerName="registry-server" containerID="cri-o://6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0" gracePeriod=2 Jul 02 22:11:49.981430 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:49.981384 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" podStartSLOduration=1.430837222 podStartE2EDuration="2.981366205s" totalImagesPullingTime="1.550528983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:11:49.980667584 +0000 UTC m=+454.964879668" watchObservedRunningTime="2026-07-02 22:11:49.981366205 +0000 UTC m=+454.965578277" Jul 02 22:11:50.003104 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.003049 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" podStartSLOduration=0.846619557 podStartE2EDuration="3.003031185s" totalImagesPullingTime="2.156411628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:11:50.000886798 +0000 UTC m=+454.985098880" watchObservedRunningTime="2026-07-02 22:11:50.003031185 +0000 UTC m=+454.987243265" Jul 02 22:11:50.216924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.216856 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:50.228139 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.228100 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r856b\" (UniqueName: \"kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b\") pod \"df96f069-d19a-47e7-ad3d-6ee097336137\" (UID: \"df96f069-d19a-47e7-ad3d-6ee097336137\") " Jul 02 22:11:50.230350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.230325 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b" (OuterVolumeSpecName: "kube-api-access-r856b") pod "df96f069-d19a-47e7-ad3d-6ee097336137" (UID: "df96f069-d19a-47e7-ad3d-6ee097336137"). InnerVolumeSpecName "kube-api-access-r856b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:11:50.329687 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.329644 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r856b\" (UniqueName: \"kubernetes.io/projected/df96f069-d19a-47e7-ad3d-6ee097336137-kube-api-access-r856b\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:11:50.970931 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.970896 2583 generic.go:356] "Generic (PLEG): container finished" podID="df96f069-d19a-47e7-ad3d-6ee097336137" containerID="6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0" exitCode=0 Jul 02 22:11:50.971435 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.970940 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" Jul 02 22:11:50.971435 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.970972 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" event={"ID":"df96f069-d19a-47e7-ad3d-6ee097336137","Type":"ContainerDied","Data":"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0"} Jul 02 22:11:50.971435 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.971008 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r6vfm" event={"ID":"df96f069-d19a-47e7-ad3d-6ee097336137","Type":"ContainerDied","Data":"8974efc45127804f566a3ff6f983093923cd5f027f4524cf7747c04af8dc0d9d"} Jul 02 22:11:50.971435 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.971026 2583 scope.go:117] "RemoveContainer" containerID="6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0" Jul 02 22:11:50.980637 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.980616 2583 scope.go:117] "RemoveContainer" containerID="6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0" Jul 02 22:11:50.980900 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:11:50.980879 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0\": container with ID starting with 6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0 not found: ID does not exist" containerID="6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0" Jul 02 22:11:50.980950 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.980908 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0"} err="failed to get container status \"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0\": rpc error: code = NotFound desc = could not find container \"6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0\": container with ID starting with 6892de3967d885b042bd807bf3132f5592f33ad1b096a1778be4bfc65e2a33a0 not found: ID does not exist" Jul 02 22:11:50.994623 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.994595 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:50.999924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:50.999901 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r6vfm"] Jul 02 22:11:51.601710 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:51.601673 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df96f069-d19a-47e7-ad3d-6ee097336137" path="/var/lib/kubelet/pods/df96f069-d19a-47e7-ad3d-6ee097336137/volumes" Jul 02 22:11:57.928883 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:57.928844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:57.929426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:57.928915 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:57.952683 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:57.952649 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:11:58.030852 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:11:58.030826 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h87w4" Jul 02 22:12:02.393905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.393860 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz"] Jul 02 22:12:02.394357 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.394243 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df96f069-d19a-47e7-ad3d-6ee097336137" containerName="registry-server" Jul 02 22:12:02.394357 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.394253 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="df96f069-d19a-47e7-ad3d-6ee097336137" containerName="registry-server" Jul 02 22:12:02.394357 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.394333 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="df96f069-d19a-47e7-ad3d-6ee097336137" containerName="registry-server" Jul 02 22:12:02.398935 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.398917 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.401947 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.401930 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sw775\"" Jul 02 22:12:02.408278 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.408250 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz"] Jul 02 22:12:02.438457 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.438416 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2p92\" (UniqueName: \"kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.438457 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.438468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.438727 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.438498 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.539633 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.539597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2p92\" (UniqueName: \"kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.539838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.539640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.539838 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.539671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.540077 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.540058 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.540158 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.540092 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.548601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.548575 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2p92\" (UniqueName: \"kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.708441 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.708407 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:02.855864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.855834 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz"] Jul 02 22:12:02.857831 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:02.857802 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3484ec_859d_4e84_949e_00bd477765a4.slice/crio-08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2 WatchSource:0}: Error finding container 08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2: Status 404 returned error can't find the container with id 08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2 Jul 02 22:12:02.993867 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.993828 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9"] Jul 02 22:12:02.997591 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:02.997574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.019698 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.011041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9"] Jul 02 22:12:03.040987 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.040951 2583 generic.go:356] "Generic (PLEG): container finished" podID="ef3484ec-859d-4e84-949e-00bd477765a4" containerID="b63981c367c802895b6edac6deab1547650832bd0b5f0dda4e324ae5c5c5326c" exitCode=0 Jul 02 22:12:03.041132 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.041016 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" event={"ID":"ef3484ec-859d-4e84-949e-00bd477765a4","Type":"ContainerDied","Data":"b63981c367c802895b6edac6deab1547650832bd0b5f0dda4e324ae5c5c5326c"} Jul 02 22:12:03.041132 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.041069 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" event={"ID":"ef3484ec-859d-4e84-949e-00bd477765a4","Type":"ContainerStarted","Data":"08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2"} Jul 02 22:12:03.044515 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.044488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.044619 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.044568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5jbz\" (UniqueName: \"kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.044619 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.044594 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.145426 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.145395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.145610 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.145555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5jbz\" (UniqueName: \"kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.145610 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.145580 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.145788 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.145769 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.145971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.145954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.155352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.155327 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5jbz\" (UniqueName: \"kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.342395 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.342271 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:03.489005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.488979 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9"] Jul 02 22:12:03.490565 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:03.490522 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d4033a_90e1_4454_8b83_5de8df918074.slice/crio-f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346 WatchSource:0}: Error finding container f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346: Status 404 returned error can't find the container with id f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346 Jul 02 22:12:03.610832 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.610775 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q"] Jul 02 22:12:03.614412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.614391 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.623796 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.623767 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q"] Jul 02 22:12:03.751587 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.751541 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.751587 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.751587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcbpq\" (UniqueName: \"kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.751804 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.751633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.852665 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.852618 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.852812 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.852670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcbpq\" (UniqueName: \"kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.852812 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.852729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.853085 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.853066 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.853129 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.853070 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.861910 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.861883 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcbpq\" (UniqueName: \"kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.964649 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.964614 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:03.996098 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:03.996059 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b"] Jul 02 22:12:04.001284 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.001256 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.010954 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.010927 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b"] Jul 02 22:12:04.048905 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.048870 2583 generic.go:356] "Generic (PLEG): container finished" podID="ef3484ec-859d-4e84-949e-00bd477765a4" containerID="97d47ebd58e1edc85f378c1b15988d2c894d6d3d6ab5d30eb570ac063a14ea9c" exitCode=0 Jul 02 22:12:04.049079 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.048947 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" event={"ID":"ef3484ec-859d-4e84-949e-00bd477765a4","Type":"ContainerDied","Data":"97d47ebd58e1edc85f378c1b15988d2c894d6d3d6ab5d30eb570ac063a14ea9c"} Jul 02 22:12:04.050719 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.050698 2583 generic.go:356] "Generic (PLEG): container finished" podID="68d4033a-90e1-4454-8b83-5de8df918074" containerID="ded90e2b0fe837ef77f23dfef76f3dce789863f5dd3358b79cf2db1b5a05135b" exitCode=0 Jul 02 22:12:04.050820 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.050782 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" event={"ID":"68d4033a-90e1-4454-8b83-5de8df918074","Type":"ContainerDied","Data":"ded90e2b0fe837ef77f23dfef76f3dce789863f5dd3358b79cf2db1b5a05135b"} Jul 02 22:12:04.050876 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.050832 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" event={"ID":"68d4033a-90e1-4454-8b83-5de8df918074","Type":"ContainerStarted","Data":"f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346"} Jul 02 22:12:04.054376 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.054351 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.054455 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.054385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsht9\" (UniqueName: \"kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.054455 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.054407 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.117696 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.117670 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q"] Jul 02 22:12:04.120541 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:04.120513 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f9c877_f581_48ea_bb9b_1dd054f2fe52.slice/crio-6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20 WatchSource:0}: Error finding container 6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20: Status 404 returned error can't find the container with id 6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20 Jul 02 22:12:04.155867 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.155831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.156053 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.155977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsht9\" (UniqueName: \"kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.156053 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.156014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.156222 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.156204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.156419 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.156403 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.165578 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.165552 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsht9\" (UniqueName: \"kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.312734 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.312698 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:04.455947 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:04.455923 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b"] Jul 02 22:12:04.457712 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:04.457680 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6934abb5_755d_4c90_9761_06345c3fdbcd.slice/crio-603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5 WatchSource:0}: Error finding container 603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5: Status 404 returned error can't find the container with id 603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5 Jul 02 22:12:05.058404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.058369 2583 generic.go:356] "Generic (PLEG): container finished" podID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerID="61325d1b4796589d2d610a293a3cabfedc1765ecb52aff2629e574d3e7897c60" exitCode=0 Jul 02 22:12:05.058806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.058449 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" event={"ID":"31f9c877-f581-48ea-bb9b-1dd054f2fe52","Type":"ContainerDied","Data":"61325d1b4796589d2d610a293a3cabfedc1765ecb52aff2629e574d3e7897c60"} Jul 02 22:12:05.058806 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.058509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" event={"ID":"31f9c877-f581-48ea-bb9b-1dd054f2fe52","Type":"ContainerStarted","Data":"6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20"} Jul 02 22:12:05.060894 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.060761 2583 generic.go:356] "Generic (PLEG): container finished" podID="ef3484ec-859d-4e84-949e-00bd477765a4" containerID="a26c09b015d1edec1c7088942e421b7c2a175804afbe19a41c5dd9c07539a81a" exitCode=0 Jul 02 22:12:05.060894 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.060835 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" event={"ID":"ef3484ec-859d-4e84-949e-00bd477765a4","Type":"ContainerDied","Data":"a26c09b015d1edec1c7088942e421b7c2a175804afbe19a41c5dd9c07539a81a"} Jul 02 22:12:05.062548 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.062523 2583 generic.go:356] "Generic (PLEG): container finished" podID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerID="7bdb3b45d322575d243607449bbcae1b25e339f3c2d7f6f381921f42e4e8c606" exitCode=0 Jul 02 22:12:05.062670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.062583 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" event={"ID":"6934abb5-755d-4c90-9761-06345c3fdbcd","Type":"ContainerDied","Data":"7bdb3b45d322575d243607449bbcae1b25e339f3c2d7f6f381921f42e4e8c606"} Jul 02 22:12:05.062670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.062643 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" event={"ID":"6934abb5-755d-4c90-9761-06345c3fdbcd","Type":"ContainerStarted","Data":"603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5"} Jul 02 22:12:05.064585 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.064565 2583 generic.go:356] "Generic (PLEG): container finished" podID="68d4033a-90e1-4454-8b83-5de8df918074" containerID="e2a1575ef139e65c33debf5bdd239f1b32119b380cc99dc768e4a68be231e54e" exitCode=0 Jul 02 22:12:05.064672 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.064614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" event={"ID":"68d4033a-90e1-4454-8b83-5de8df918074","Type":"ContainerDied","Data":"e2a1575ef139e65c33debf5bdd239f1b32119b380cc99dc768e4a68be231e54e"} Jul 02 22:12:05.527209 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:05.527162 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:06.072438 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.072399 2583 generic.go:356] "Generic (PLEG): container finished" podID="68d4033a-90e1-4454-8b83-5de8df918074" containerID="966b663451bc8e1cb17e16bd16efb3da97009ef8229eeea1b08ecabfde2626b0" exitCode=0 Jul 02 22:12:06.072823 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.072479 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" event={"ID":"68d4033a-90e1-4454-8b83-5de8df918074","Type":"ContainerDied","Data":"966b663451bc8e1cb17e16bd16efb3da97009ef8229eeea1b08ecabfde2626b0"} Jul 02 22:12:06.074275 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.074247 2583 generic.go:356] "Generic (PLEG): container finished" podID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerID="860738e30b8f54a401d3e477b83dc0e39751196b6e3419d73f63bd49ddda73fd" exitCode=0 Jul 02 22:12:06.074404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.074322 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" event={"ID":"6934abb5-755d-4c90-9761-06345c3fdbcd","Type":"ContainerDied","Data":"860738e30b8f54a401d3e477b83dc0e39751196b6e3419d73f63bd49ddda73fd"} Jul 02 22:12:06.208070 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.207650 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:06.272839 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.272789 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2p92\" (UniqueName: \"kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92\") pod \"ef3484ec-859d-4e84-949e-00bd477765a4\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " Jul 02 22:12:06.272974 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.272892 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util\") pod \"ef3484ec-859d-4e84-949e-00bd477765a4\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " Jul 02 22:12:06.272974 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.272928 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle\") pod \"ef3484ec-859d-4e84-949e-00bd477765a4\" (UID: \"ef3484ec-859d-4e84-949e-00bd477765a4\") " Jul 02 22:12:06.273634 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.273592 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle" (OuterVolumeSpecName: "bundle") pod "ef3484ec-859d-4e84-949e-00bd477765a4" (UID: "ef3484ec-859d-4e84-949e-00bd477765a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:06.275005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.274983 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92" (OuterVolumeSpecName: "kube-api-access-x2p92") pod "ef3484ec-859d-4e84-949e-00bd477765a4" (UID: "ef3484ec-859d-4e84-949e-00bd477765a4"). InnerVolumeSpecName "kube-api-access-x2p92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:06.278021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.277997 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util" (OuterVolumeSpecName: "util") pod "ef3484ec-859d-4e84-949e-00bd477765a4" (UID: "ef3484ec-859d-4e84-949e-00bd477765a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:06.374366 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.374235 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2p92\" (UniqueName: \"kubernetes.io/projected/ef3484ec-859d-4e84-949e-00bd477765a4-kube-api-access-x2p92\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:06.374366 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.374268 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:06.374366 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:06.374277 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef3484ec-859d-4e84-949e-00bd477765a4-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:07.082023 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.081988 2583 generic.go:356] "Generic (PLEG): container finished" podID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerID="31922e73f75bad124a716631be7900d97f89259fae574c222b696d7ba4f08179" exitCode=0 Jul 02 22:12:07.082505 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.082077 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" event={"ID":"31f9c877-f581-48ea-bb9b-1dd054f2fe52","Type":"ContainerDied","Data":"31922e73f75bad124a716631be7900d97f89259fae574c222b696d7ba4f08179"} Jul 02 22:12:07.084230 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.084213 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" Jul 02 22:12:07.084386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.084213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz" event={"ID":"ef3484ec-859d-4e84-949e-00bd477765a4","Type":"ContainerDied","Data":"08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2"} Jul 02 22:12:07.084386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.084347 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08b49dd2fb2b309307cc007887a6de6a2a7972e61b51e14f8477ebe3e67dc4f2" Jul 02 22:12:07.086597 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.086573 2583 generic.go:356] "Generic (PLEG): container finished" podID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerID="5c3cbc16dd2e9bbb6e6e4d48c4cd86e3af7d6bf9e9e7cd94d4b55ce29f7a2f7b" exitCode=0 Jul 02 22:12:07.086691 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.086643 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" event={"ID":"6934abb5-755d-4c90-9761-06345c3fdbcd","Type":"ContainerDied","Data":"5c3cbc16dd2e9bbb6e6e4d48c4cd86e3af7d6bf9e9e7cd94d4b55ce29f7a2f7b"} Jul 02 22:12:07.288611 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.288584 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:07.383711 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.383618 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5jbz\" (UniqueName: \"kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz\") pod \"68d4033a-90e1-4454-8b83-5de8df918074\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " Jul 02 22:12:07.383711 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.383680 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle\") pod \"68d4033a-90e1-4454-8b83-5de8df918074\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " Jul 02 22:12:07.383711 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.383708 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util\") pod \"68d4033a-90e1-4454-8b83-5de8df918074\" (UID: \"68d4033a-90e1-4454-8b83-5de8df918074\") " Jul 02 22:12:07.384296 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.384260 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle" (OuterVolumeSpecName: "bundle") pod "68d4033a-90e1-4454-8b83-5de8df918074" (UID: "68d4033a-90e1-4454-8b83-5de8df918074"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:07.385784 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.385761 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz" (OuterVolumeSpecName: "kube-api-access-b5jbz") pod "68d4033a-90e1-4454-8b83-5de8df918074" (UID: "68d4033a-90e1-4454-8b83-5de8df918074"). InnerVolumeSpecName "kube-api-access-b5jbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:07.389213 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.389177 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util" (OuterVolumeSpecName: "util") pod "68d4033a-90e1-4454-8b83-5de8df918074" (UID: "68d4033a-90e1-4454-8b83-5de8df918074"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:07.485203 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.485144 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5jbz\" (UniqueName: \"kubernetes.io/projected/68d4033a-90e1-4454-8b83-5de8df918074-kube-api-access-b5jbz\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:07.485203 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.485195 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:07.485203 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:07.485206 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68d4033a-90e1-4454-8b83-5de8df918074-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:08.094356 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.094324 2583 generic.go:356] "Generic (PLEG): container finished" podID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerID="5a6be90cce7c1db781187254f5808394a8d03ba81bed5a7f0e06540417ea8bd7" exitCode=0 Jul 02 22:12:08.094815 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.094412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" event={"ID":"31f9c877-f581-48ea-bb9b-1dd054f2fe52","Type":"ContainerDied","Data":"5a6be90cce7c1db781187254f5808394a8d03ba81bed5a7f0e06540417ea8bd7"} Jul 02 22:12:08.096349 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.096330 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" Jul 02 22:12:08.096478 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.096326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9" event={"ID":"68d4033a-90e1-4454-8b83-5de8df918074","Type":"ContainerDied","Data":"f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346"} Jul 02 22:12:08.096478 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.096434 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f83d00a49d11b4ba99230839dbaf2c44a53f6ccdc1c2770c128bfc91659e0346" Jul 02 22:12:08.234458 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.234429 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:08.293128 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.293084 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsht9\" (UniqueName: \"kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9\") pod \"6934abb5-755d-4c90-9761-06345c3fdbcd\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " Jul 02 22:12:08.293346 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.293155 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util\") pod \"6934abb5-755d-4c90-9761-06345c3fdbcd\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " Jul 02 22:12:08.293346 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.293172 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle\") pod \"6934abb5-755d-4c90-9761-06345c3fdbcd\" (UID: \"6934abb5-755d-4c90-9761-06345c3fdbcd\") " Jul 02 22:12:08.293856 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.293828 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle" (OuterVolumeSpecName: "bundle") pod "6934abb5-755d-4c90-9761-06345c3fdbcd" (UID: "6934abb5-755d-4c90-9761-06345c3fdbcd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:08.295460 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.295433 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9" (OuterVolumeSpecName: "kube-api-access-rsht9") pod "6934abb5-755d-4c90-9761-06345c3fdbcd" (UID: "6934abb5-755d-4c90-9761-06345c3fdbcd"). InnerVolumeSpecName "kube-api-access-rsht9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:08.298680 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.298658 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util" (OuterVolumeSpecName: "util") pod "6934abb5-755d-4c90-9761-06345c3fdbcd" (UID: "6934abb5-755d-4c90-9761-06345c3fdbcd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:08.394551 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.394456 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:08.394551 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.394487 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6934abb5-755d-4c90-9761-06345c3fdbcd-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:08.394551 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:08.394499 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsht9\" (UniqueName: \"kubernetes.io/projected/6934abb5-755d-4c90-9761-06345c3fdbcd-kube-api-access-rsht9\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.103775 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.103744 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" Jul 02 22:12:09.103775 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.103763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b" event={"ID":"6934abb5-755d-4c90-9761-06345c3fdbcd","Type":"ContainerDied","Data":"603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5"} Jul 02 22:12:09.104200 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.103837 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="603662c1ce115f0609b8cf52052537d55841d7bd72a006044c3c2f2d34d8a6b5" Jul 02 22:12:09.243347 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.243320 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:09.302601 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.302562 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle\") pod \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " Jul 02 22:12:09.302776 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.302668 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcbpq\" (UniqueName: \"kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq\") pod \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " Jul 02 22:12:09.302776 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.302699 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util\") pod \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\" (UID: \"31f9c877-f581-48ea-bb9b-1dd054f2fe52\") " Jul 02 22:12:09.303075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.303038 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle" (OuterVolumeSpecName: "bundle") pod "31f9c877-f581-48ea-bb9b-1dd054f2fe52" (UID: "31f9c877-f581-48ea-bb9b-1dd054f2fe52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.304801 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.304766 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq" (OuterVolumeSpecName: "kube-api-access-vcbpq") pod "31f9c877-f581-48ea-bb9b-1dd054f2fe52" (UID: "31f9c877-f581-48ea-bb9b-1dd054f2fe52"). InnerVolumeSpecName "kube-api-access-vcbpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:09.308275 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.308237 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util" (OuterVolumeSpecName: "util") pod "31f9c877-f581-48ea-bb9b-1dd054f2fe52" (UID: "31f9c877-f581-48ea-bb9b-1dd054f2fe52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.404227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.404134 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcbpq\" (UniqueName: \"kubernetes.io/projected/31f9c877-f581-48ea-bb9b-1dd054f2fe52-kube-api-access-vcbpq\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.404227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.404167 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-util\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.404227 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:09.404176 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31f9c877-f581-48ea-bb9b-1dd054f2fe52-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:10.111712 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:10.111684 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" Jul 02 22:12:10.112119 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:10.111680 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q" event={"ID":"31f9c877-f581-48ea-bb9b-1dd054f2fe52","Type":"ContainerDied","Data":"6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20"} Jul 02 22:12:10.112119 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:10.111813 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b514962a9b098c1f8a0df9744a77cb63cd612cd78f7ee32530d90e81bde0f20" Jul 02 22:12:15.910938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:15.910840 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:19.349693 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:19.349649 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:20.714217 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714181 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb"] Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714585 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="pull" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714597 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="pull" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714610 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="util" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714616 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="util" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714624 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="extract" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714629 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="extract" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714637 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="util" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714642 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="util" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714648 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="pull" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714653 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="pull" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714660 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="extract" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714666 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="extract" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714677 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="extract" Jul 02 22:12:20.714675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714682 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714688 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="pull" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714693 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="pull" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714701 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="pull" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714706 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="pull" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714716 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="util" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714721 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="util" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714728 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="util" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714732 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="util" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714739 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714743 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714805 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef3484ec-859d-4e84-949e-00bd477765a4" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714813 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6934abb5-755d-4c90-9761-06345c3fdbcd" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714818 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f9c877-f581-48ea-bb9b-1dd054f2fe52" containerName="extract" Jul 02 22:12:20.715107 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.714826 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="68d4033a-90e1-4454-8b83-5de8df918074" containerName="extract" Jul 02 22:12:20.723090 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.723067 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.726871 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.726845 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-67bdg\"" Jul 02 22:12:20.737740 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.737710 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb"] Jul 02 22:12:20.810570 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.810529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.810760 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.810605 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x4gz\" (UniqueName: \"kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.911121 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.911081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x4gz\" (UniqueName: \"kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.911339 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.911178 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.911603 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.911578 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.920906 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.920877 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x4gz\" (UniqueName: \"kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:20.927786 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:20.927758 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:21.034199 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:21.034101 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:21.187475 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:21.187440 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb"] Jul 02 22:12:21.190962 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:21.190933 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd46c0e7f_686d_49c4_b651_88e241cd5c43.slice/crio-4fff1fa8aea3f023df8ec345a0f36e726f78ba2c354d2f0a8a67eb0f62922a47 WatchSource:0}: Error finding container 4fff1fa8aea3f023df8ec345a0f36e726f78ba2c354d2f0a8a67eb0f62922a47: Status 404 returned error can't find the container with id 4fff1fa8aea3f023df8ec345a0f36e726f78ba2c354d2f0a8a67eb0f62922a47 Jul 02 22:12:21.301423 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:21.301329 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:22.181390 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:22.181351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" event={"ID":"d46c0e7f-686d-49c4-b651-88e241cd5c43","Type":"ContainerStarted","Data":"4fff1fa8aea3f023df8ec345a0f36e726f78ba2c354d2f0a8a67eb0f62922a47"} Jul 02 22:12:22.400179 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:22.400131 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:22.505636 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:22.505533 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:23.311545 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.311510 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz"] Jul 02 22:12:23.315489 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.315470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:23.324690 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.324669 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p9g6x\"" Jul 02 22:12:23.333627 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.333599 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz"] Jul 02 22:12:23.433927 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.433895 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp5wb\" (UniqueName: \"kubernetes.io/projected/fd853420-5605-41e9-8499-09cdc7506e26-kube-api-access-wp5wb\") pod \"authorino-operator-75cc8d5cb7-7tnpz\" (UID: \"fd853420-5605-41e9-8499-09cdc7506e26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:23.535218 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.535182 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp5wb\" (UniqueName: \"kubernetes.io/projected/fd853420-5605-41e9-8499-09cdc7506e26-kube-api-access-wp5wb\") pod \"authorino-operator-75cc8d5cb7-7tnpz\" (UID: \"fd853420-5605-41e9-8499-09cdc7506e26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:23.553657 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.553614 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp5wb\" (UniqueName: \"kubernetes.io/projected/fd853420-5605-41e9-8499-09cdc7506e26-kube-api-access-wp5wb\") pod \"authorino-operator-75cc8d5cb7-7tnpz\" (UID: \"fd853420-5605-41e9-8499-09cdc7506e26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:23.626062 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.625960 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:23.792556 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.792523 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz"] Jul 02 22:12:23.798705 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:23.798662 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd853420_5605_41e9_8499_09cdc7506e26.slice/crio-34c199c86ec525a56ee396a9947a067406d24d3095629e4391d2db784f7039a2 WatchSource:0}: Error finding container 34c199c86ec525a56ee396a9947a067406d24d3095629e4391d2db784f7039a2: Status 404 returned error can't find the container with id 34c199c86ec525a56ee396a9947a067406d24d3095629e4391d2db784f7039a2 Jul 02 22:12:23.945939 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:23.945891 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:24.045600 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:24.045565 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:24.195784 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:24.195748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" event={"ID":"fd853420-5605-41e9-8499-09cdc7506e26","Type":"ContainerStarted","Data":"34c199c86ec525a56ee396a9947a067406d24d3095629e4391d2db784f7039a2"} Jul 02 22:12:25.620267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.620229 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq"] Jul 02 22:12:25.625720 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.625697 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:25.629375 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.629351 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 22:12:25.629516 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.629353 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-l6bf6\"" Jul 02 22:12:25.640023 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.639993 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq"] Jul 02 22:12:25.755379 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.755328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kk7j\" (UniqueName: \"kubernetes.io/projected/e8d0d185-a762-4e8a-87d4-8bbb149ffc20-kube-api-access-8kk7j\") pod \"dns-operator-controller-manager-864dd78c4c-728gq\" (UID: \"e8d0d185-a762-4e8a-87d4-8bbb149ffc20\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:25.856513 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.856467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kk7j\" (UniqueName: \"kubernetes.io/projected/e8d0d185-a762-4e8a-87d4-8bbb149ffc20-kube-api-access-8kk7j\") pod \"dns-operator-controller-manager-864dd78c4c-728gq\" (UID: \"e8d0d185-a762-4e8a-87d4-8bbb149ffc20\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:25.869386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.869349 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kk7j\" (UniqueName: \"kubernetes.io/projected/e8d0d185-a762-4e8a-87d4-8bbb149ffc20-kube-api-access-8kk7j\") pod \"dns-operator-controller-manager-864dd78c4c-728gq\" (UID: \"e8d0d185-a762-4e8a-87d4-8bbb149ffc20\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:25.940290 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:25.940248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:26.114646 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.114583 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:26.409022 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.408926 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc797b768-qcg9l"] Jul 02 22:12:26.412891 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.412864 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.428235 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.428196 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc797b768-qcg9l"] Jul 02 22:12:26.499185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.499153 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:26.563764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.563728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-service-ca\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.563943 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.563826 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvjtq\" (UniqueName: \"kubernetes.io/projected/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-kube-api-access-lvjtq\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.563943 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.563864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-trusted-ca-bundle\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.563943 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.563907 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-oauth-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.564118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.563984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.564118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.564017 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.564118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.564063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-oauth-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.594243 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.594171 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:26.664992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.664897 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.664992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.664950 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.664992 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.664992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-oauth-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665566 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-service-ca\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665566 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665155 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvjtq\" (UniqueName: \"kubernetes.io/projected/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-kube-api-access-lvjtq\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665566 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-trusted-ca-bundle\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665566 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-oauth-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665811 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-oauth-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.665929 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.665907 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-service-ca\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.666355 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.666330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.666759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.666736 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-trusted-ca-bundle\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.667948 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.667926 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-serving-cert\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.668035 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.667926 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-console-oauth-config\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.674982 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.674957 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvjtq\" (UniqueName: \"kubernetes.io/projected/6bcb00a8-1c80-4fdc-ac52-8445bbecdea6-kube-api-access-lvjtq\") pod \"console-bc797b768-qcg9l\" (UID: \"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6\") " pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.695958 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.695925 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:26.727450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.727367 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:26.794407 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:26.794283 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:27.203667 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.203629 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:27.309348 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.309297 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:27.486506 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.486477 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc797b768-qcg9l"] Jul 02 22:12:27.487656 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:27.487623 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcb00a8_1c80_4fdc_ac52_8445bbecdea6.slice/crio-cedf471a9ffa364f60aab7990c114bd2319ff10755319692de289f3b3457b487 WatchSource:0}: Error finding container cedf471a9ffa364f60aab7990c114bd2319ff10755319692de289f3b3457b487: Status 404 returned error can't find the container with id cedf471a9ffa364f60aab7990c114bd2319ff10755319692de289f3b3457b487 Jul 02 22:12:27.500501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.500478 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq"] Jul 02 22:12:27.501368 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:27.501346 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d0d185_a762_4e8a_87d4_8bbb149ffc20.slice/crio-3ae6a3d155fd7ef131d7ca47b5b21f640d90fc2a8d318d30416428b6568547f7 WatchSource:0}: Error finding container 3ae6a3d155fd7ef131d7ca47b5b21f640d90fc2a8d318d30416428b6568547f7: Status 404 returned error can't find the container with id 3ae6a3d155fd7ef131d7ca47b5b21f640d90fc2a8d318d30416428b6568547f7 Jul 02 22:12:27.709271 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.709234 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck"] Jul 02 22:12:27.712997 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.712972 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:27.715889 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.715856 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mmz57\"" Jul 02 22:12:27.722318 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.722278 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck"] Jul 02 22:12:27.877068 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.876979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jrc\" (UniqueName: \"kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc\") pod \"limitador-operator-controller-manager-859d7b78bd-bfcck\" (UID: \"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:27.977577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:27.977535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jrc\" (UniqueName: \"kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc\") pod \"limitador-operator-controller-manager-859d7b78bd-bfcck\" (UID: \"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:28.002095 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.002056 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jrc\" (UniqueName: \"kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc\") pod \"limitador-operator-controller-manager-859d7b78bd-bfcck\" (UID: \"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:28.024372 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.024340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:28.064509 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.064470 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:28.189223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.189193 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck"] Jul 02 22:12:28.196781 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:28.196743 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8ff489_b1e8_4cfa_84b1_e7eaca37ce6e.slice/crio-7bd91632e83be84356ad45303273e78d79413570adf72e1bef03a0f005f67b61 WatchSource:0}: Error finding container 7bd91632e83be84356ad45303273e78d79413570adf72e1bef03a0f005f67b61: Status 404 returned error can't find the container with id 7bd91632e83be84356ad45303273e78d79413570adf72e1bef03a0f005f67b61 Jul 02 22:12:28.230835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.230799 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" event={"ID":"e8d0d185-a762-4e8a-87d4-8bbb149ffc20","Type":"ContainerStarted","Data":"3ae6a3d155fd7ef131d7ca47b5b21f640d90fc2a8d318d30416428b6568547f7"} Jul 02 22:12:28.235379 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.235332 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" event={"ID":"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e","Type":"ContainerStarted","Data":"7bd91632e83be84356ad45303273e78d79413570adf72e1bef03a0f005f67b61"} Jul 02 22:12:28.237622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.237584 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc797b768-qcg9l" event={"ID":"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6","Type":"ContainerStarted","Data":"48edc3a780c9fa024942621388d9497899fd1d0071863db4c5bf4c4d773994b0"} Jul 02 22:12:28.237790 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.237658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc797b768-qcg9l" event={"ID":"6bcb00a8-1c80-4fdc-ac52-8445bbecdea6","Type":"ContainerStarted","Data":"cedf471a9ffa364f60aab7990c114bd2319ff10755319692de289f3b3457b487"} Jul 02 22:12:28.240180 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.240147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" event={"ID":"d46c0e7f-686d-49c4-b651-88e241cd5c43","Type":"ContainerStarted","Data":"6f87d222cbf1c9daea482b3da7eac4fed00e18c409d3fe6805d3216580831620"} Jul 02 22:12:28.240379 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.240362 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:28.242386 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.242360 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" event={"ID":"fd853420-5605-41e9-8499-09cdc7506e26","Type":"ContainerStarted","Data":"49a2ea818c72238926563a642db50ce5a9a625e783b2d555a7b7165600bd5be2"} Jul 02 22:12:28.242523 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.242508 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:28.264894 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.264830 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bc797b768-qcg9l" podStartSLOduration=2.264803828 podStartE2EDuration="2.264803828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:28.26119212 +0000 UTC m=+493.245404202" watchObservedRunningTime="2026-07-02 22:12:28.264803828 +0000 UTC m=+493.249015910" Jul 02 22:12:28.284458 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.284327 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" podStartSLOduration=1.766083214 podStartE2EDuration="5.28429209s" totalImagesPullingTime="3.518208876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:28.281581943 +0000 UTC m=+493.265794024" watchObservedRunningTime="2026-07-02 22:12:28.28429209 +0000 UTC m=+493.268504171" Jul 02 22:12:28.308267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:28.308202 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" podStartSLOduration=2.164210421 podStartE2EDuration="8.308179665s" totalImagesPullingTime="6.143969244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:28.30365937 +0000 UTC m=+493.287871451" watchObservedRunningTime="2026-07-02 22:12:28.308179665 +0000 UTC m=+493.292391745" Jul 02 22:12:29.712832 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:29.712796 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:29.813451 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:29.813421 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:30.262379 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:30.262341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" event={"ID":"e8d0d185-a762-4e8a-87d4-8bbb149ffc20","Type":"ContainerStarted","Data":"640157ea4726186b723546b8f90035b70ae39d01e7f3c55aab932759fc8c47f5"} Jul 02 22:12:30.262594 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:30.262417 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:30.295759 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:30.295705 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" podStartSLOduration=3.076385068 podStartE2EDuration="5.295687305s" totalImagesPullingTime="2.219302237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:30.293636218 +0000 UTC m=+495.277848312" watchObservedRunningTime="2026-07-02 22:12:30.295687305 +0000 UTC m=+495.279899386" Jul 02 22:12:30.480536 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:30.480505 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:30.771699 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:30.771662 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:31.270028 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:31.269984 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" event={"ID":"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e","Type":"ContainerStarted","Data":"9f7d2e3f95d7cdb4eced7fe2fe72975574adf0f388073ab8e404d44cb00a600e"} Jul 02 22:12:31.270208 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:31.270128 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:31.290795 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:31.290738 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" podStartSLOduration=2.064738293 podStartE2EDuration="4.290719902s" totalImagesPullingTime="2.225981609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:31.287836062 +0000 UTC m=+496.272048143" watchObservedRunningTime="2026-07-02 22:12:31.290719902 +0000 UTC m=+496.274931982" Jul 02 22:12:36.728376 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:36.728332 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:36.728777 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:36.728436 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:36.733499 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:36.733473 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:37.308427 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:37.308397 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc797b768-qcg9l" Jul 02 22:12:37.373669 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:37.373631 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:12:37.515580 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:37.515548 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:37.615929 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:37.615835 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:37.716018 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:37.715983 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:38.918757 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:38.918720 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:39.019495 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:39.019452 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:39.254019 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:39.253991 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7tnpz" Jul 02 22:12:39.254224 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:39.254072 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.182577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.182538 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb"] Jul 02 22:12:41.182967 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.182777 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" containerName="manager" containerID="cri-o://6f87d222cbf1c9daea482b3da7eac4fed00e18c409d3fe6805d3216580831620" gracePeriod=2 Jul 02 22:12:41.185586 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.185547 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.188267 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.188241 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb"] Jul 02 22:12:41.213063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.213029 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck"] Jul 02 22:12:41.213281 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.213250 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" containerName="manager" containerID="cri-o://9f7d2e3f95d7cdb4eced7fe2fe72975574adf0f388073ab8e404d44cb00a600e" gracePeriod=2 Jul 02 22:12:41.215481 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.215461 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.222034 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.222004 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:12:41.222581 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.222561 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" containerName="manager" Jul 02 22:12:41.222581 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.222581 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" containerName="manager" Jul 02 22:12:41.222737 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.222714 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" containerName="manager" Jul 02 22:12:41.226352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.226330 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck"] Jul 02 22:12:41.226479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.226402 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.228913 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.228881 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.241475 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.241438 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:12:41.252099 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.252069 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w"] Jul 02 22:12:41.252576 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.252557 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" containerName="manager" Jul 02 22:12:41.252682 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.252578 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" containerName="manager" Jul 02 22:12:41.252739 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.252719 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" containerName="manager" Jul 02 22:12:41.255922 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.255901 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:41.264146 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.264105 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bfcck\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.267016 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.266982 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.273080 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.273056 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-728gq" Jul 02 22:12:41.278761 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.278734 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w"] Jul 02 22:12:41.295055 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.295023 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bfcck\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.297518 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.297490 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.300035 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.300009 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.302611 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.302583 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bfcck\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.305073 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.305052 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.305183 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.305170 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ggb\" (UniqueName: \"kubernetes.io/projected/79e74c3c-8320-4e5a-b40d-11e6b3992ccc-kube-api-access-59ggb\") pod \"limitador-operator-controller-manager-859d7b78bd-vzn4w\" (UID: \"79e74c3c-8320-4e5a-b40d-11e6b3992ccc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:41.305287 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.305218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcwtz\" (UniqueName: \"kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.331400 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.331361 2583 generic.go:356] "Generic (PLEG): container finished" podID="d46c0e7f-686d-49c4-b651-88e241cd5c43" containerID="6f87d222cbf1c9daea482b3da7eac4fed00e18c409d3fe6805d3216580831620" exitCode=0 Jul 02 22:12:41.333857 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.333825 2583 generic.go:356] "Generic (PLEG): container finished" podID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" containerID="9f7d2e3f95d7cdb4eced7fe2fe72975574adf0f388073ab8e404d44cb00a600e" exitCode=0 Jul 02 22:12:41.406226 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.406183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ggb\" (UniqueName: \"kubernetes.io/projected/79e74c3c-8320-4e5a-b40d-11e6b3992ccc-kube-api-access-59ggb\") pod \"limitador-operator-controller-manager-859d7b78bd-vzn4w\" (UID: \"79e74c3c-8320-4e5a-b40d-11e6b3992ccc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:41.406395 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.406284 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcwtz\" (UniqueName: \"kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.406473 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.406420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.406864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.406838 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.421689 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.421657 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ggb\" (UniqueName: \"kubernetes.io/projected/79e74c3c-8320-4e5a-b40d-11e6b3992ccc-kube-api-access-59ggb\") pod \"limitador-operator-controller-manager-859d7b78bd-vzn4w\" (UID: \"79e74c3c-8320-4e5a-b40d-11e6b3992ccc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:41.423201 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.423171 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcwtz\" (UniqueName: \"kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9dbz\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.431702 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.431660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:41.469021 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.468992 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.471197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.471178 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.471750 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.471721 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bfcck\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.474121 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.474100 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.476491 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.476471 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bfcck\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:41.478827 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:12:41.478803 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v5glb\" is forbidden: User \"system:node:ip-10-0-128-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-202.ec2.internal' and this object" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:41.507341 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.507281 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume\") pod \"d46c0e7f-686d-49c4-b651-88e241cd5c43\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " Jul 02 22:12:41.507341 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.507347 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x4gz\" (UniqueName: \"kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz\") pod \"d46c0e7f-686d-49c4-b651-88e241cd5c43\" (UID: \"d46c0e7f-686d-49c4-b651-88e241cd5c43\") " Jul 02 22:12:41.507583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.507469 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4jrc\" (UniqueName: \"kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc\") pod \"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e\" (UID: \"2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e\") " Jul 02 22:12:41.507882 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.507850 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d46c0e7f-686d-49c4-b651-88e241cd5c43" (UID: "d46c0e7f-686d-49c4-b651-88e241cd5c43"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:41.509517 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.509496 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz" (OuterVolumeSpecName: "kube-api-access-8x4gz") pod "d46c0e7f-686d-49c4-b651-88e241cd5c43" (UID: "d46c0e7f-686d-49c4-b651-88e241cd5c43"). InnerVolumeSpecName "kube-api-access-8x4gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:41.509581 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.509559 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc" (OuterVolumeSpecName: "kube-api-access-g4jrc") pod "2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" (UID: "2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e"). InnerVolumeSpecName "kube-api-access-g4jrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:41.533610 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.533572 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:41.601971 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.601939 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e" path="/var/lib/kubelet/pods/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e/volumes" Jul 02 22:12:41.602312 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.602288 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d46c0e7f-686d-49c4-b651-88e241cd5c43" path="/var/lib/kubelet/pods/d46c0e7f-686d-49c4-b651-88e241cd5c43/volumes" Jul 02 22:12:41.608133 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.608107 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4jrc\" (UniqueName: \"kubernetes.io/projected/2e8ff489-b1e8-4cfa-84b1-e7eaca37ce6e-kube-api-access-g4jrc\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:41.608234 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.608136 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d46c0e7f-686d-49c4-b651-88e241cd5c43-extensions-socket-volume\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:41.608234 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.608158 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x4gz\" (UniqueName: \"kubernetes.io/projected/d46c0e7f-686d-49c4-b651-88e241cd5c43-kube-api-access-8x4gz\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:12:41.617440 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.617417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:41.624659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.624635 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:41.818188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.818159 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w"] Jul 02 22:12:41.819517 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:41.819494 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e74c3c_8320_4e5a_b40d_11e6b3992ccc.slice/crio-b5fedc18f603d4957b8bcf2f1ec7162f0df9e7f5921698b8829779493359f1a0 WatchSource:0}: Error finding container b5fedc18f603d4957b8bcf2f1ec7162f0df9e7f5921698b8829779493359f1a0: Status 404 returned error can't find the container with id b5fedc18f603d4957b8bcf2f1ec7162f0df9e7f5921698b8829779493359f1a0 Jul 02 22:12:41.829504 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:41.829483 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:12:41.830496 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:12:41.830469 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8e5437_4e60_4b1a_9b1c_545437f9dff5.slice/crio-4b1c2778c4c0c68781a99d2677c21b6f9cc0ce4037d5319fd8c5d42d53fbd294 WatchSource:0}: Error finding container 4b1c2778c4c0c68781a99d2677c21b6f9cc0ce4037d5319fd8c5d42d53fbd294: Status 404 returned error can't find the container with id 4b1c2778c4c0c68781a99d2677c21b6f9cc0ce4037d5319fd8c5d42d53fbd294 Jul 02 22:12:42.341782 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.341730 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" event={"ID":"79e74c3c-8320-4e5a-b40d-11e6b3992ccc","Type":"ContainerStarted","Data":"eea170dc235bafda3a8b94c680bf8985ca74e53334ad8b08e518a09f86874cda"} Jul 02 22:12:42.342242 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.341802 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" event={"ID":"79e74c3c-8320-4e5a-b40d-11e6b3992ccc","Type":"ContainerStarted","Data":"b5fedc18f603d4957b8bcf2f1ec7162f0df9e7f5921698b8829779493359f1a0"} Jul 02 22:12:42.342242 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.341834 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:42.343622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.343593 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" event={"ID":"4f8e5437-4e60-4b1a-9b1c-545437f9dff5","Type":"ContainerStarted","Data":"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c"} Jul 02 22:12:42.343772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.343648 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" event={"ID":"4f8e5437-4e60-4b1a-9b1c-545437f9dff5","Type":"ContainerStarted","Data":"4b1c2778c4c0c68781a99d2677c21b6f9cc0ce4037d5319fd8c5d42d53fbd294"} Jul 02 22:12:42.343772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.343681 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:42.345004 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.344983 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfcck" Jul 02 22:12:42.345113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.345045 2583 scope.go:117] "RemoveContainer" containerID="9f7d2e3f95d7cdb4eced7fe2fe72975574adf0f388073ab8e404d44cb00a600e" Jul 02 22:12:42.346789 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.346757 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v5glb" Jul 02 22:12:42.355265 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.355230 2583 scope.go:117] "RemoveContainer" containerID="6f87d222cbf1c9daea482b3da7eac4fed00e18c409d3fe6805d3216580831620" Jul 02 22:12:42.396818 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.396765 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" podStartSLOduration=1.396746865 podStartE2EDuration="1.396746865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:42.39363867 +0000 UTC m=+507.377850751" watchObservedRunningTime="2026-07-02 22:12:42.396746865 +0000 UTC m=+507.380958946" Jul 02 22:12:42.454667 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:42.454612 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" podStartSLOduration=1.454595473 podStartE2EDuration="1.454595473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:42.45163515 +0000 UTC m=+507.435847230" watchObservedRunningTime="2026-07-02 22:12:42.454595473 +0000 UTC m=+507.438807554" Jul 02 22:12:46.127291 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:46.127255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:46.229777 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:46.229739 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:49.929829 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:49.929789 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:50.026187 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:50.026144 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:53.356973 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:53.356942 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vzn4w" Jul 02 22:12:53.357516 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:53.357021 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:12:58.725857 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:58.725818 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:12:58.825544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:12:58.825508 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:02.332237 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.332189 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:02.396494 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.396448 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f469f9f4d-v4z8w" podUID="8eade025-036b-48d6-bc07-82b7f0d8acf9" containerName="console" containerID="cri-o://08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b" gracePeriod=15 Jul 02 22:13:02.432063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.432024 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:02.653544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.653516 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:13:02.807143 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807102 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807143 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807145 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807437 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807171 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807437 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807229 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdb7s\" (UniqueName: \"kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807437 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807248 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807437 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807286 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807648 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807457 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert\") pod \"8eade025-036b-48d6-bc07-82b7f0d8acf9\" (UID: \"8eade025-036b-48d6-bc07-82b7f0d8acf9\") " Jul 02 22:13:02.807648 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807631 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config" (OuterVolumeSpecName: "console-config") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:13:02.807753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807706 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca" (OuterVolumeSpecName: "service-ca") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:13:02.807753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807720 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:13:02.807950 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807930 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:13:02.807997 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807971 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.807997 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.807988 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-service-ca\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.808080 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.808000 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-trusted-ca-bundle\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.809527 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.809501 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:13:02.809955 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.809935 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:13:02.810029 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.809954 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s" (OuterVolumeSpecName: "kube-api-access-pdb7s") pod "8eade025-036b-48d6-bc07-82b7f0d8acf9" (UID: "8eade025-036b-48d6-bc07-82b7f0d8acf9"). InnerVolumeSpecName "kube-api-access-pdb7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:13:02.908531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.908441 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdb7s\" (UniqueName: \"kubernetes.io/projected/8eade025-036b-48d6-bc07-82b7f0d8acf9-kube-api-access-pdb7s\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.908531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.908471 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.908531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.908484 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eade025-036b-48d6-bc07-82b7f0d8acf9-oauth-serving-cert\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:02.908531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:02.908498 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eade025-036b-48d6-bc07-82b7f0d8acf9-console-oauth-config\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:03.485197 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.485164 2583 generic.go:356] "Generic (PLEG): container finished" podID="8eade025-036b-48d6-bc07-82b7f0d8acf9" containerID="08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b" exitCode=0 Jul 02 22:13:03.485624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.485240 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f469f9f4d-v4z8w" Jul 02 22:13:03.485624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.485250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f469f9f4d-v4z8w" event={"ID":"8eade025-036b-48d6-bc07-82b7f0d8acf9","Type":"ContainerDied","Data":"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b"} Jul 02 22:13:03.485624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.485368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f469f9f4d-v4z8w" event={"ID":"8eade025-036b-48d6-bc07-82b7f0d8acf9","Type":"ContainerDied","Data":"a7c780781b5ca11d4ed63cf7e36fb130ac6aa58b5d1b11780bc22dde8b8031f4"} Jul 02 22:13:03.485624 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.485398 2583 scope.go:117] "RemoveContainer" containerID="08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b" Jul 02 22:13:03.496241 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.496217 2583 scope.go:117] "RemoveContainer" containerID="08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b" Jul 02 22:13:03.496580 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:13:03.496562 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b\": container with ID starting with 08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b not found: ID does not exist" containerID="08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b" Jul 02 22:13:03.496638 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.496588 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b"} err="failed to get container status \"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b\": rpc error: code = NotFound desc = could not find container \"08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b\": container with ID starting with 08985a307344f0d105ba410aa50c7a4c3130089a4e69501962b91af435b2cb3b not found: ID does not exist" Jul 02 22:13:03.509357 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.509293 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:13:03.514448 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.514419 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f469f9f4d-v4z8w"] Jul 02 22:13:03.602440 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:03.602402 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eade025-036b-48d6-bc07-82b7f0d8acf9" path="/var/lib/kubelet/pods/8eade025-036b-48d6-bc07-82b7f0d8acf9/volumes" Jul 02 22:13:07.321600 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:07.321516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:07.458670 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:07.458632 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:11.148394 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:11.148348 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:11.249562 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:11.249532 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:11.759193 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:11.759151 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:13:11.759463 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:11.759439 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" podUID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" containerName="manager" containerID="cri-o://d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c" gracePeriod=10 Jul 02 22:13:12.028860 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.028832 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:13:12.196908 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.196865 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcwtz\" (UniqueName: \"kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz\") pod \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " Jul 02 22:13:12.197294 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.196933 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume\") pod \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\" (UID: \"4f8e5437-4e60-4b1a-9b1c-545437f9dff5\") " Jul 02 22:13:12.197403 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.197377 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f8e5437-4e60-4b1a-9b1c-545437f9dff5" (UID: "4f8e5437-4e60-4b1a-9b1c-545437f9dff5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:13:12.199030 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.199010 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz" (OuterVolumeSpecName: "kube-api-access-dcwtz") pod "4f8e5437-4e60-4b1a-9b1c-545437f9dff5" (UID: "4f8e5437-4e60-4b1a-9b1c-545437f9dff5"). InnerVolumeSpecName "kube-api-access-dcwtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:13:12.297990 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.297896 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcwtz\" (UniqueName: \"kubernetes.io/projected/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-kube-api-access-dcwtz\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:12.297990 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.297927 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8e5437-4e60-4b1a-9b1c-545437f9dff5-extensions-socket-volume\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:12.540250 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.540216 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" containerID="d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c" exitCode=0 Jul 02 22:13:12.540450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.540268 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" Jul 02 22:13:12.540450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.540320 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" event={"ID":"4f8e5437-4e60-4b1a-9b1c-545437f9dff5","Type":"ContainerDied","Data":"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c"} Jul 02 22:13:12.540450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.540376 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz" event={"ID":"4f8e5437-4e60-4b1a-9b1c-545437f9dff5","Type":"ContainerDied","Data":"4b1c2778c4c0c68781a99d2677c21b6f9cc0ce4037d5319fd8c5d42d53fbd294"} Jul 02 22:13:12.540450 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.540399 2583 scope.go:117] "RemoveContainer" containerID="d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c" Jul 02 22:13:12.550391 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.550369 2583 scope.go:117] "RemoveContainer" containerID="d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c" Jul 02 22:13:12.550682 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:13:12.550663 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c\": container with ID starting with d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c not found: ID does not exist" containerID="d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c" Jul 02 22:13:12.550745 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.550689 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c"} err="failed to get container status \"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c\": rpc error: code = NotFound desc = could not find container \"d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c\": container with ID starting with d198273775fb788b7a6edc3f647f3b470e9a74a6751ecc07b2bfd4a57baa6f9c not found: ID does not exist" Jul 02 22:13:12.565479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.565441 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:13:12.571005 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:12.570977 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9dbz"] Jul 02 22:13:13.602119 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:13.602087 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" path="/var/lib/kubelet/pods/4f8e5437-4e60-4b1a-9b1c-545437f9dff5/volumes" Jul 02 22:13:22.520118 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:22.520072 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:22.620577 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:22.620532 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:23.117930 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:23.117892 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:23.221802 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:23.221758 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:24.329927 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:24.329893 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:24.431067 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:24.431022 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:28.149870 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.149831 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79"] Jul 02 22:13:28.150352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150338 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eade025-036b-48d6-bc07-82b7f0d8acf9" containerName="console" Jul 02 22:13:28.150352 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150354 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eade025-036b-48d6-bc07-82b7f0d8acf9" containerName="console" Jul 02 22:13:28.150449 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150370 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" containerName="manager" Jul 02 22:13:28.150449 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150376 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" containerName="manager" Jul 02 22:13:28.150449 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150439 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f8e5437-4e60-4b1a-9b1c-545437f9dff5" containerName="manager" Jul 02 22:13:28.150449 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.150446 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eade025-036b-48d6-bc07-82b7f0d8acf9" containerName="console" Jul 02 22:13:28.155002 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.154978 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.158360 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.158328 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7zgmv\"" Jul 02 22:13:28.158494 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.158422 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 22:13:28.169583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.169555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79"] Jul 02 22:13:28.177463 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.177428 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:28.240724 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.240724 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240725 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.240724 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdtc7\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-kube-api-access-vdtc7\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.240986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.241015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.241008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.341823 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.341785 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.341823 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.341826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.341851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.341988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342083 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342244 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342244 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342405 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342260 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342405 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342349 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342405 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdtc7\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-kube-api-access-vdtc7\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342578 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342411 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342578 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342502 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342687 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342569 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.342977 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.342956 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.344290 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.344264 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.344406 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.344367 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.351953 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.351914 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.352063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.352041 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdtc7\" (UniqueName: \"kubernetes.io/projected/a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2-kube-api-access-vdtc7\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5g79\" (UID: \"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.466239 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.466204 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:28.573894 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.573815 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:28.630936 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.630894 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79"] Jul 02 22:13:28.632133 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:13:28.632108 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bb7557_40e8_4e81_b0e9_5bc2ba843ac2.slice/crio-d3573b88a791ee9f66f3d20573b07aefbf473fbc68361179b5b2291f68989916 WatchSource:0}: Error finding container d3573b88a791ee9f66f3d20573b07aefbf473fbc68361179b5b2291f68989916: Status 404 returned error can't find the container with id d3573b88a791ee9f66f3d20573b07aefbf473fbc68361179b5b2291f68989916 Jul 02 22:13:28.925015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:28.924902 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:29.044254 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:29.044219 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:29.641764 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:29.641727 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" event={"ID":"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2","Type":"ContainerStarted","Data":"d3573b88a791ee9f66f3d20573b07aefbf473fbc68361179b5b2291f68989916"} Jul 02 22:13:32.137009 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.136973 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 02 22:13:32.137340 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.137044 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 02 22:13:32.137340 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.137077 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 02 22:13:32.442534 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.442492 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:32.663256 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.663219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" event={"ID":"a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2","Type":"ContainerStarted","Data":"ce827ead51aeaa507b644947fa148d6a7574a8a981cdca090b230846332257dd"} Jul 02 22:13:32.687019 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:32.686962 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" podStartSLOduration=1.1844620319999999 podStartE2EDuration="4.686941472s" totalImagesPullingTime="3.50247944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:13:32.684431943 +0000 UTC m=+557.668644026" watchObservedRunningTime="2026-07-02 22:13:32.686941472 +0000 UTC m=+557.671153618" Jul 02 22:13:33.466735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.466698 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:33.471707 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.471673 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:33.669981 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.669941 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:33.670941 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.670919 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5g79" Jul 02 22:13:33.726401 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.726268 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:33.730255 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.730234 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.733733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.733706 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sw775\"" Jul 02 22:13:33.733733 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.733722 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 22:13:33.741849 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.741821 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:33.743824 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.743793 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:33.788943 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.788905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.789136 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.788997 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltmtw\" (UniqueName: \"kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.823506 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.823463 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:33.825317 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.825268 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:33.889479 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.889434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.889675 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.889531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltmtw\" (UniqueName: \"kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.890060 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.890040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:33.891461 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.891436 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:33.898224 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:33.898194 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltmtw\" (UniqueName: \"kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw\") pod \"limitador-limitador-64c94fd64f-8j67x\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:34.038901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.038798 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p2545"] Jul 02 22:13:34.042748 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.042727 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.043750 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.043730 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:34.052996 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.052966 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p2545"] Jul 02 22:13:34.082122 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.082086 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p2545"] Jul 02 22:13:34.091414 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.091379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdlwr\" (UniqueName: \"kubernetes.io/projected/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-kube-api-access-tdlwr\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.091628 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.091604 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-config-file\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.192766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.192726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-config-file\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.192937 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.192837 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdlwr\" (UniqueName: \"kubernetes.io/projected/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-kube-api-access-tdlwr\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.193540 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.193520 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-config-file\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.199295 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.199264 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:34.200561 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:13:34.200533 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11802040_952e_4a64_9678_e1c3d43b832c.slice/crio-59d8fff1a524e51732b155e3391706960c1df8eac36903a15391bd1de8734db6 WatchSource:0}: Error finding container 59d8fff1a524e51732b155e3391706960c1df8eac36903a15391bd1de8734db6: Status 404 returned error can't find the container with id 59d8fff1a524e51732b155e3391706960c1df8eac36903a15391bd1de8734db6 Jul 02 22:13:34.202958 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.202930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdlwr\" (UniqueName: \"kubernetes.io/projected/8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c-kube-api-access-tdlwr\") pod \"limitador-limitador-54d6d988d5-p2545\" (UID: \"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.355404 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.355293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:34.508557 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.508527 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p2545"] Jul 02 22:13:34.509850 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:13:34.509824 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5c8d8f_e18d_4d71_bbca_cf0aaecd7f7c.slice/crio-ea69197013e606ca8861fb8cfa0e9c714b3cc551a9410fbfc43889c0f29efb60 WatchSource:0}: Error finding container ea69197013e606ca8861fb8cfa0e9c714b3cc551a9410fbfc43889c0f29efb60: Status 404 returned error can't find the container with id ea69197013e606ca8861fb8cfa0e9c714b3cc551a9410fbfc43889c0f29efb60 Jul 02 22:13:34.557923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.557890 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:34.630116 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.630033 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:34.677165 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.677133 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" event={"ID":"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c","Type":"ContainerStarted","Data":"ea69197013e606ca8861fb8cfa0e9c714b3cc551a9410fbfc43889c0f29efb60"} Jul 02 22:13:34.678365 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:34.678339 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" event={"ID":"11802040-952e-4a64-9678-e1c3d43b832c","Type":"ContainerStarted","Data":"59d8fff1a524e51732b155e3391706960c1df8eac36903a15391bd1de8734db6"} Jul 02 22:13:37.700322 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.700269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" event={"ID":"11802040-952e-4a64-9678-e1c3d43b832c","Type":"ContainerStarted","Data":"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d"} Jul 02 22:13:37.700756 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.700458 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:37.701946 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.701918 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" event={"ID":"8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c","Type":"ContainerStarted","Data":"d200ec276ae62cfd259ac8cb629d69796c5a5ff1ba4f953dd0d8762e4271b471"} Jul 02 22:13:37.702100 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.701976 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:37.721744 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.721679 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" podStartSLOduration=1.70141425 podStartE2EDuration="4.721665448s" totalImagesPullingTime="3.020251198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:13:37.718432536 +0000 UTC m=+562.702644618" watchObservedRunningTime="2026-07-02 22:13:37.721665448 +0000 UTC m=+562.705877529" Jul 02 22:13:37.742180 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:37.742124 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" podStartSLOduration=1.02256564 podStartE2EDuration="3.74210867s" totalImagesPullingTime="2.71954303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:13:37.740797245 +0000 UTC m=+562.725009327" watchObservedRunningTime="2026-07-02 22:13:37.74210867 +0000 UTC m=+562.726320791" Jul 02 22:13:48.708321 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:48.708271 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p2545" Jul 02 22:13:48.708793 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:48.708387 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:48.794185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:48.794142 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:48.794429 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:48.794362 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" podUID="11802040-952e-4a64-9678-e1c3d43b832c" containerName="limitador" containerID="cri-o://e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d" gracePeriod=30 Jul 02 22:13:49.359533 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.359471 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:49.442571 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.442537 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltmtw\" (UniqueName: \"kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw\") pod \"11802040-952e-4a64-9678-e1c3d43b832c\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " Jul 02 22:13:49.442778 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.442636 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file\") pod \"11802040-952e-4a64-9678-e1c3d43b832c\" (UID: \"11802040-952e-4a64-9678-e1c3d43b832c\") " Jul 02 22:13:49.443003 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.442973 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file" (OuterVolumeSpecName: "config-file") pod "11802040-952e-4a64-9678-e1c3d43b832c" (UID: "11802040-952e-4a64-9678-e1c3d43b832c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:13:49.444761 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.444725 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw" (OuterVolumeSpecName: "kube-api-access-ltmtw") pod "11802040-952e-4a64-9678-e1c3d43b832c" (UID: "11802040-952e-4a64-9678-e1c3d43b832c"). InnerVolumeSpecName "kube-api-access-ltmtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:13:49.544383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.544337 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltmtw\" (UniqueName: \"kubernetes.io/projected/11802040-952e-4a64-9678-e1c3d43b832c-kube-api-access-ltmtw\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:49.544383 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.544374 2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/11802040-952e-4a64-9678-e1c3d43b832c-config-file\") on node \"ip-10-0-128-202.ec2.internal\" DevicePath \"\"" Jul 02 22:13:49.774417 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.774384 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" Jul 02 22:13:49.774879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.774417 2583 generic.go:356] "Generic (PLEG): container finished" podID="11802040-952e-4a64-9678-e1c3d43b832c" containerID="e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d" exitCode=0 Jul 02 22:13:49.774879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.774388 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" event={"ID":"11802040-952e-4a64-9678-e1c3d43b832c","Type":"ContainerDied","Data":"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d"} Jul 02 22:13:49.774879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.774521 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8j67x" event={"ID":"11802040-952e-4a64-9678-e1c3d43b832c","Type":"ContainerDied","Data":"59d8fff1a524e51732b155e3391706960c1df8eac36903a15391bd1de8734db6"} Jul 02 22:13:49.774879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.774544 2583 scope.go:117] "RemoveContainer" containerID="e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d" Jul 02 22:13:49.784735 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.784715 2583 scope.go:117] "RemoveContainer" containerID="e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d" Jul 02 22:13:49.785019 ip-10-0-128-202 kubenswrapper[2583]: E0702 22:13:49.784999 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d\": container with ID starting with e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d not found: ID does not exist" containerID="e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d" Jul 02 22:13:49.785116 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.785025 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d"} err="failed to get container status \"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d\": rpc error: code = NotFound desc = could not find container \"e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d\": container with ID starting with e194081ef904759f014021da65de55e47b19ff7937f5cffefb737a8f77bdec4d not found: ID does not exist" Jul 02 22:13:49.792924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.792899 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:49.798377 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:49.798349 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8j67x"] Jul 02 22:13:50.194785 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.194750 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-fq2k4"] Jul 02 22:13:50.195211 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.195197 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11802040-952e-4a64-9678-e1c3d43b832c" containerName="limitador" Jul 02 22:13:50.195211 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.195212 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11802040-952e-4a64-9678-e1c3d43b832c" containerName="limitador" Jul 02 22:13:50.195340 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.195283 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11802040-952e-4a64-9678-e1c3d43b832c" containerName="limitador" Jul 02 22:13:50.199734 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.199706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.202865 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.202837 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 22:13:50.203007 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.202837 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d7d5b\"" Jul 02 22:13:50.212116 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.212090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-fq2k4"] Jul 02 22:13:50.223193 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.223159 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:50.251229 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.251188 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca48b42d-5c02-49df-bfcb-1007efe59047-data\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.251229 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.251235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lck7w\" (UniqueName: \"kubernetes.io/projected/ca48b42d-5c02-49df-bfcb-1007efe59047-kube-api-access-lck7w\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.352680 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.352638 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca48b42d-5c02-49df-bfcb-1007efe59047-data\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.352680 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.352684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lck7w\" (UniqueName: \"kubernetes.io/projected/ca48b42d-5c02-49df-bfcb-1007efe59047-kube-api-access-lck7w\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.353048 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.353027 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca48b42d-5c02-49df-bfcb-1007efe59047-data\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.362288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.362253 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lck7w\" (UniqueName: \"kubernetes.io/projected/ca48b42d-5c02-49df-bfcb-1007efe59047-kube-api-access-lck7w\") pod \"postgres-69fb4594cf-fq2k4\" (UID: \"ca48b42d-5c02-49df-bfcb-1007efe59047\") " pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.510346 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.510212 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:13:50.660659 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.660637 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-fq2k4"] Jul 02 22:13:50.662509 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:13:50.662478 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca48b42d_5c02_49df_bfcb_1007efe59047.slice/crio-096ec6b6ab0d2109ecdebb4db1e7f18471e1bfbbe879918f7c4297956031ac26 WatchSource:0}: Error finding container 096ec6b6ab0d2109ecdebb4db1e7f18471e1bfbbe879918f7c4297956031ac26: Status 404 returned error can't find the container with id 096ec6b6ab0d2109ecdebb4db1e7f18471e1bfbbe879918f7c4297956031ac26 Jul 02 22:13:50.782381 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:50.782284 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-fq2k4" event={"ID":"ca48b42d-5c02-49df-bfcb-1007efe59047","Type":"ContainerStarted","Data":"096ec6b6ab0d2109ecdebb4db1e7f18471e1bfbbe879918f7c4297956031ac26"} Jul 02 22:13:51.603901 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:13:51.603858 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11802040-952e-4a64-9678-e1c3d43b832c" path="/var/lib/kubelet/pods/11802040-952e-4a64-9678-e1c3d43b832c/volumes" Jul 02 22:14:06.887835 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:06.887791 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-fq2k4" event={"ID":"ca48b42d-5c02-49df-bfcb-1007efe59047","Type":"ContainerStarted","Data":"0a4afbae5b52d86239483ba1402daef6503bed2a4cc6c727d1daab36e2767c2d"} Jul 02 22:14:06.888259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:06.887867 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:14:06.907390 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:06.907334 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-fq2k4" podStartSLOduration=1.443255546 podStartE2EDuration="16.90731807s" totalImagesPullingTime="15.464062524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:14:06.90502341 +0000 UTC m=+591.889235494" watchObservedRunningTime="2026-07-02 22:14:06.90731807 +0000 UTC m=+591.891530158" Jul 02 22:14:12.925613 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:12.925584 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-fq2k4" Jul 02 22:14:13.262960 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:13.262917 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:13.363181 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:13.363150 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:15.506492 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:15.506468 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:14:15.506928 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:15.506791 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:14:19.492467 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:19.492423 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:19.636073 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:19.636039 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:19.714330 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:19.714273 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:22.524786 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:22.524740 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:22.651474 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:22.651435 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:23.611466 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:23.611430 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:49.888530 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:49.888440 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:51.299465 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:51.299434 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:58.354254 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:14:58.354214 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:03.560288 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:15:03.560243 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:34.350773 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:15:34.350738 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:34.450695 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:15:34.450661 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:35.593046 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:16:35.593006 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:18:04.687167 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:18:04.687072 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:19:15.554485 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:19:15.554459 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:19:15.554948 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:19:15.554793 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:22:05.649375 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:22:05.649338 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:24:15.593904 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:15.593816 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:24:15.597509 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:15.597484 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:24:45.120588 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.120554 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jkb7n_0c1bc535-d935-492f-bbe3-ab279cfd0356/manager/0.log" Jul 02 22:24:45.231277 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.231243 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r7bw5_34ac77d9-965e-4bf0-9dbd-7886dff6add5/manager/0.log" Jul 02 22:24:45.472709 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.472681 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4qrvg_ff5bb769-5627-4a49-ad70-5f368cfd9438/server/0.log" Jul 02 22:24:45.595565 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.595527 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5ws9s_92593e17-87ff-41b4-8577-8d94d0c6c608/manager/0.log" Jul 02 22:24:45.945753 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.945722 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b66c9c9d4-vrtcx_3fef31d2-ca5c-43ac-bfdf-6f59b569c040/manager/0.log" Jul 02 22:24:45.951428 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:45.951406 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b66c9c9d4-vrtcx_3fef31d2-ca5c-43ac-bfdf-6f59b569c040/copy-manifests/0.log" Jul 02 22:24:46.063760 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.063734 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-fq2k4_ca48b42d-5c02-49df-bfcb-1007efe59047/postgres/0.log" Jul 02 22:24:46.837816 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.837784 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/util/0.log" Jul 02 22:24:46.844772 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.844742 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/pull/0.log" Jul 02 22:24:46.851702 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.851675 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/extract/0.log" Jul 02 22:24:46.965545 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.965514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/util/0.log" Jul 02 22:24:46.972744 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.972719 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/pull/0.log" Jul 02 22:24:46.979185 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:46.979164 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/extract/0.log" Jul 02 22:24:47.083127 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.083097 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/util/0.log" Jul 02 22:24:47.089946 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.089876 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/pull/0.log" Jul 02 22:24:47.096295 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.096274 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/extract/0.log" Jul 02 22:24:47.210501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.210473 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/util/0.log" Jul 02 22:24:47.218247 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.218218 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/pull/0.log" Jul 02 22:24:47.225770 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.225739 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/extract/0.log" Jul 02 22:24:47.438938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.438905 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7tnpz_fd853420-5605-41e9-8499-09cdc7506e26/manager/0.log" Jul 02 22:24:47.548340 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.548288 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-728gq_e8d0d185-a762-4e8a-87d4-8bbb149ffc20/manager/0.log" Jul 02 22:24:47.783532 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:47.783428 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h87w4_73f347cf-0daf-407b-ad58-f7499b7d7b55/registry-server/0.log" Jul 02 22:24:48.012990 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:48.012957 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p2545_8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c/limitador/0.log" Jul 02 22:24:48.120821 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:48.120737 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vzn4w_79e74c3c-8320-4e5a-b40d-11e6b3992ccc/manager/0.log" Jul 02 22:24:48.784364 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:48.784335 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f57454dd4-zk8tl_44aded33-eedb-4d47-aa97-780abfb2e300/kube-auth-proxy/0.log" Jul 02 22:24:48.900054 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:48.900024 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5g79_a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2/istio-proxy/0.log" Jul 02 22:24:49.018363 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:49.018336 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d48d4fd89-qjr8q_bd81f4e8-4237-4f44-a4ea-8886d1bf8441/router/0.log" Jul 02 22:24:53.312514 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:53.312485 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:24:53.386150 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:53.386115 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" limit=1024 Jul 02 22:24:56.140410 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:56.140378 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q55pk_0c995aee-a8c5-4d10-b49e-47da30a93d9b/global-pull-secret-syncer/0.log" Jul 02 22:24:56.194328 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:56.194285 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7sz94_be2968ab-3e35-4ac0-9403-378a0f2fe535/konnectivity-agent/0.log" Jul 02 22:24:56.272265 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:24:56.272227 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-202.ec2.internal_948f3bc00963a2485050a8f8925dd13e/haproxy/0.log" Jul 02 22:25:00.663205 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.663177 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/extract/0.log" Jul 02 22:25:00.702017 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.701991 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/util/0.log" Jul 02 22:25:00.737802 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.737713 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qx88q_31f9c877-f581-48ea-bb9b-1dd054f2fe52/pull/0.log" Jul 02 22:25:00.775652 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.775614 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/extract/0.log" Jul 02 22:25:00.802579 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.802553 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/util/0.log" Jul 02 22:25:00.828412 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.828385 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ll6n9_68d4033a-90e1-4454-8b83-5de8df918074/pull/0.log" Jul 02 22:25:00.859848 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.859815 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/extract/0.log" Jul 02 22:25:00.882745 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.882720 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/util/0.log" Jul 02 22:25:00.906513 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.906481 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cdh8b_6934abb5-755d-4c90-9761-06345c3fdbcd/pull/0.log" Jul 02 22:25:00.935622 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.935596 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/extract/0.log" Jul 02 22:25:00.961342 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.961296 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/util/0.log" Jul 02 22:25:00.985223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:00.985191 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mb9mz_ef3484ec-859d-4e84-949e-00bd477765a4/pull/0.log" Jul 02 22:25:01.050544 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:01.050447 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7tnpz_fd853420-5605-41e9-8499-09cdc7506e26/manager/0.log" Jul 02 22:25:01.076599 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:01.076570 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-728gq_e8d0d185-a762-4e8a-87d4-8bbb149ffc20/manager/0.log" Jul 02 22:25:01.149592 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:01.149547 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h87w4_73f347cf-0daf-407b-ad58-f7499b7d7b55/registry-server/0.log" Jul 02 22:25:01.208696 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:01.208662 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p2545_8c5c8d8f-e18d-4d71-bbca-cf0aaecd7f7c/limitador/0.log" Jul 02 22:25:01.245607 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:01.245558 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vzn4w_79e74c3c-8320-4e5a-b40d-11e6b3992ccc/manager/0.log" Jul 02 22:25:02.827088 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.827061 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/alertmanager/0.log" Jul 02 22:25:02.850665 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.850635 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/config-reloader/0.log" Jul 02 22:25:02.876405 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.876373 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/kube-rbac-proxy-web/0.log" Jul 02 22:25:02.899879 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.899847 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/kube-rbac-proxy/0.log" Jul 02 22:25:02.925924 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.925877 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/kube-rbac-proxy-metric/0.log" Jul 02 22:25:02.956822 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.956788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/prom-label-proxy/0.log" Jul 02 22:25:02.979792 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:02.979761 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a303012c-f3ed-42de-803b-794a64f4248b/init-config-reloader/0.log" Jul 02 22:25:03.027796 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.027765 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-c7crx_855a99f7-2255-47c6-831b-8b9eebbfc19c/cluster-monitoring-operator/0.log" Jul 02 22:25:03.213582 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.213556 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76958644b7-8rnf7_b24ebc97-8c8e-486a-83f7-83b177197cd7/metrics-server/0.log" Jul 02 22:25:03.298025 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.297998 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pmzx_11a10297-7920-4ee0-9159-74cbcf53b708/node-exporter/0.log" Jul 02 22:25:03.323531 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.323505 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pmzx_11a10297-7920-4ee0-9159-74cbcf53b708/kube-rbac-proxy/0.log" Jul 02 22:25:03.347504 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.347480 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pmzx_11a10297-7920-4ee0-9159-74cbcf53b708/init-textfile/0.log" Jul 02 22:25:03.968726 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:03.968697 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-85nrh_d6a7c966-eaa4-4fb6-a3fa-ba4ff46e13e4/prometheus-operator-admission-webhook/0.log" Jul 02 22:25:04.006940 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:04.006911 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546c5f6498-zlzww_322e8ba0-a95f-4079-9cdc-28f1acd92284/telemeter-client/0.log" Jul 02 22:25:04.033259 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:04.033234 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546c5f6498-zlzww_322e8ba0-a95f-4079-9cdc-28f1acd92284/reload/0.log" Jul 02 22:25:04.058415 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:04.058382 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546c5f6498-zlzww_322e8ba0-a95f-4079-9cdc-28f1acd92284/kube-rbac-proxy/0.log" Jul 02 22:25:05.489876 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:05.489847 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-pk6n9_518c9c14-f70e-45a0-a566-80b717650204/networking-console-plugin/0.log" Jul 02 22:25:06.160938 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.160901 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/2.log" Jul 02 22:25:06.168290 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.168262 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-gd55z_3ae65cac-b4f8-4b54-944b-b6fd61f251c0/console-operator/3.log" Jul 02 22:25:06.355350 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.355313 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v"] Jul 02 22:25:06.359188 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.359170 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.362162 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.362130 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6qm4t\"/\"default-dockercfg-mxrcs\"" Jul 02 22:25:06.363389 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.363370 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qm4t\"/\"openshift-service-ca.crt\"" Jul 02 22:25:06.363497 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.363396 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qm4t\"/\"kube-root-ca.crt\"" Jul 02 22:25:06.378840 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.378806 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v"] Jul 02 22:25:06.391899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.391866 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7sn9\" (UniqueName: \"kubernetes.io/projected/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-kube-api-access-n7sn9\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.392075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.391967 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-proc\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.392075 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.392008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-sys\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.392229 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.392119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-lib-modules\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.392229 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.392160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-podres\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-proc\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493066 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-sys\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-lib-modules\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-podres\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493152 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-proc\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493161 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7sn9\" (UniqueName: \"kubernetes.io/projected/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-kube-api-access-n7sn9\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493263 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-sys\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493324 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-podres\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.493583 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.493337 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-lib-modules\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.502864 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.502839 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7sn9\" (UniqueName: \"kubernetes.io/projected/d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4-kube-api-access-n7sn9\") pod \"perf-node-gather-daemonset-pnr4v\" (UID: \"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4\") " pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.669419 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.669376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:06.748766 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.748691 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bc797b768-qcg9l_6bcb00a8-1c80-4fdc-ac52-8445bbecdea6/console/0.log" Jul 02 22:25:06.815685 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.815632 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-9m6tc_c3de5eeb-49f4-48d7-93b1-ab5140d5e407/download-server/0.log" Jul 02 22:25:06.818833 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:06.818805 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v"] Jul 02 22:25:06.820763 ip-10-0-128-202 kubenswrapper[2583]: W0702 22:25:06.820733 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7bb2b7a_7fcc_4c8c_99c7_4632ed4c3fc4.slice/crio-19afa3599199179e938f8b987d617a26b957aa121629cd34de2c6593304a79ce WatchSource:0}: Error finding container 19afa3599199179e938f8b987d617a26b957aa121629cd34de2c6593304a79ce: Status 404 returned error can't find the container with id 19afa3599199179e938f8b987d617a26b957aa121629cd34de2c6593304a79ce Jul 02 22:25:07.435639 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:07.435566 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-8f7h6_b5c221fd-0fd5-40ad-9562-a5af0216d438/volume-data-source-validator/0.log" Jul 02 22:25:07.797466 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:07.797365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" event={"ID":"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4","Type":"ContainerStarted","Data":"7b46730a60b142764a2d341dbab2bb4b85ef2a8aea8017d749c79be8ca179926"} Jul 02 22:25:07.797466 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:07.797422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" event={"ID":"d7bb2b7a-7fcc-4c8c-99c7-4632ed4c3fc4","Type":"ContainerStarted","Data":"19afa3599199179e938f8b987d617a26b957aa121629cd34de2c6593304a79ce"} Jul 02 22:25:07.797899 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:07.797565 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:07.814199 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:07.814150 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" podStartSLOduration=1.814134184 podStartE2EDuration="1.814134184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:25:07.812122719 +0000 UTC m=+1252.796334801" watchObservedRunningTime="2026-07-02 22:25:07.814134184 +0000 UTC m=+1252.798346267" Jul 02 22:25:08.543812 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:08.543778 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tpg8k_20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b/dns/0.log" Jul 02 22:25:08.565930 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:08.565904 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tpg8k_20ef0b8f-6617-4fd3-b0b4-82c2a5ca363b/kube-rbac-proxy/0.log" Jul 02 22:25:08.645498 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:08.645470 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zmxlq_02517e0f-1525-477c-bee1-3bc04b0ceebb/dns-node-resolver/0.log" Jul 02 22:25:09.274947 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:09.274917 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fqvf7_69304590-8e37-4dc2-a16a-b8e8ea9208a5/node-ca/0.log" Jul 02 22:25:10.352756 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:10.352715 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f57454dd4-zk8tl_44aded33-eedb-4d47-aa97-780abfb2e300/kube-auth-proxy/0.log" Jul 02 22:25:10.386662 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:10.386628 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5g79_a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2/istio-proxy/0.log" Jul 02 22:25:10.415015 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:10.414987 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d48d4fd89-qjr8q_bd81f4e8-4237-4f44-a4ea-8886d1bf8441/router/0.log" Jul 02 22:25:11.028358 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.028296 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxxn5_ac9ca684-7062-49c0-bbaf-5dd463e3ecfb/serve-healthcheck-canary/0.log" Jul 02 22:25:11.559090 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.559054 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-srrnc_c32d31b2-74db-42d9-adfe-24d20e516268/insights-operator/0.log" Jul 02 22:25:11.559647 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.559379 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-srrnc_c32d31b2-74db-42d9-adfe-24d20e516268/insights-operator/1.log" Jul 02 22:25:11.732098 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.732040 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5vzl_0344693d-b473-4240-a043-8a5e609a6793/kube-rbac-proxy/0.log" Jul 02 22:25:11.754673 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.754640 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5vzl_0344693d-b473-4240-a043-8a5e609a6793/exporter/0.log" Jul 02 22:25:11.777385 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:11.777356 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5vzl_0344693d-b473-4240-a043-8a5e609a6793/extractor/0.log" Jul 02 22:25:13.813033 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:13.813009 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6qm4t/perf-node-gather-daemonset-pnr4v" Jul 02 22:25:13.824785 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:13.824753 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jkb7n_0c1bc535-d935-492f-bbe3-ab279cfd0356/manager/0.log" Jul 02 22:25:13.856633 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:13.856604 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r7bw5_34ac77d9-965e-4bf0-9dbd-7886dff6add5/manager/0.log" Jul 02 22:25:13.905438 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:13.905409 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4qrvg_ff5bb769-5627-4a49-ad70-5f368cfd9438/server/0.log" Jul 02 22:25:13.971223 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:13.971191 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5ws9s_92593e17-87ff-41b4-8577-8d94d0c6c608/manager/0.log" Jul 02 22:25:14.139642 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:14.139550 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b66c9c9d4-vrtcx_3fef31d2-ca5c-43ac-bfdf-6f59b569c040/manager/0.log" Jul 02 22:25:14.163219 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:14.163190 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b66c9c9d4-vrtcx_3fef31d2-ca5c-43ac-bfdf-6f59b569c040/copy-manifests/0.log" Jul 02 22:25:14.190063 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:14.190036 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-fq2k4_ca48b42d-5c02-49df-bfcb-1007efe59047/postgres/0.log" Jul 02 22:25:15.511700 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:15.511669 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-927t8_e363df1e-eade-44e0-b06f-d9960f16547e/openshift-lws-operator/0.log" Jul 02 22:25:16.544362 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:16.544258 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f57454dd4-zk8tl_44aded33-eedb-4d47-aa97-780abfb2e300/kube-auth-proxy/0.log" Jul 02 22:25:16.563143 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:16.563116 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5g79_a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2/istio-proxy/0.log" Jul 02 22:25:16.579046 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:16.579018 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d48d4fd89-qjr8q_bd81f4e8-4237-4f44-a4ea-8886d1bf8441/router/0.log" Jul 02 22:25:17.670501 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:17.670471 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f57454dd4-zk8tl_44aded33-eedb-4d47-aa97-780abfb2e300/kube-auth-proxy/0.log" Jul 02 22:25:17.691113 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:17.691082 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5g79_a5bb7557-40e8-4e81-b0e9-5bc2ba843ac2/istio-proxy/0.log" Jul 02 22:25:17.712578 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:17.712548 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d48d4fd89-qjr8q_bd81f4e8-4237-4f44-a4ea-8886d1bf8441/router/0.log" Jul 02 22:25:21.091275 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:21.091206 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-gcwlb_9d469a43-f522-4f7c-b597-2ae7c3bff1db/kube-storage-version-migrator-operator/1.log" Jul 02 22:25:21.093001 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:21.092973 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-gcwlb_9d469a43-f522-4f7c-b597-2ae7c3bff1db/kube-storage-version-migrator-operator/0.log" Jul 02 22:25:22.236411 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.236382 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/kube-multus-additional-cni-plugins/0.log" Jul 02 22:25:22.259398 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.259366 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/egress-router-binary-copy/0.log" Jul 02 22:25:22.282713 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.282681 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/cni-plugins/0.log" Jul 02 22:25:22.305199 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.305173 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/bond-cni-plugin/0.log" Jul 02 22:25:22.327724 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.327697 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/routeoverride-cni/0.log" Jul 02 22:25:22.351958 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.351928 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/whereabouts-cni-bincopy/0.log" Jul 02 22:25:22.374804 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.374777 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llrgq_f8c728c9-da49-4fba-9184-e4c93b0b197e/whereabouts-cni/0.log" Jul 02 22:25:22.621591 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.621498 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h2qzt_fb7ce651-2b47-47dd-8464-bce148a8b057/kube-multus/0.log" Jul 02 22:25:22.722241 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.722197 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sqlzp_4a4ca393-cf0e-4866-b73a-76ad7790a42d/network-metrics-daemon/0.log" Jul 02 22:25:22.744918 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:22.744891 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sqlzp_4a4ca393-cf0e-4866-b73a-76ad7790a42d/kube-rbac-proxy/0.log" Jul 02 22:25:23.923923 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:23.923889 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chpms_c25580bb-6095-4326-91af-eef364ec7f4d/ovn-controller/0.log" Jul 02 22:25:23.943553 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:23.943524 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chpms_c25580bb-6095-4326-91af-eef364ec7f4d/ovn-acl-logging/0.log" Jul 02 22:25:23.970939 ip-10-0-128-202 kubenswrapper[2583]: I0702 22:25:23.970910 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-chpms_c25580bb-6095-4326-91af-eef364ec7f4d/kube-rbac-proxy-node/0.log"