Jul 21 12:25:29.023372 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 12:25:29.023382 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 12:25:29.023389 ip-10-0-140-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 12:25:29.023632 ip-10-0-140-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 12:25:39.057878 ip-10-0-140-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 12:25:39.057897 ip-10-0-140-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot eddc504cfbae41aeac7c54f861a4c0ee -- Jul 21 12:27:32.365570 ip-10-0-140-37 systemd[1]: Starting Kubernetes Kubelet... Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.720086 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728578 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728598 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728603 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728606 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728610 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728613 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728616 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:32.794547 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728619 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728622 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728625 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728628 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728630 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728633 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728635 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728638 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728640 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728643 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728647 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728649 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728652 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728654 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728657 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728660 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728662 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728665 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728667 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728670 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:32.795409 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728673 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728676 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728678 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728681 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728684 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728690 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728692 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728695 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728697 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728699 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728702 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728705 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728707 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728710 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728713 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728716 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728719 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728721 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728724 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728726 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:32.796100 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728728 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728732 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728738 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728742 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728745 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728752 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728755 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728758 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728761 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728763 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728767 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728771 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728774 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728777 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728780 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728783 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728785 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728788 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728791 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:32.796651 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728794 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728798 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728801 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728804 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728808 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728812 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728815 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728817 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728820 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728823 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728825 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728828 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728830 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728833 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728835 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728838 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728841 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728845 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728848 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728850 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:32.797180 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728852 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728856 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728859 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728862 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728865 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728867 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728870 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728873 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728875 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728878 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728881 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728883 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.728886 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729733 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729742 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729746 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729749 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729752 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729755 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729757 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:32.797720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729760 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729763 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729766 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729768 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729771 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729773 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729776 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729778 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729781 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729783 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729786 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729789 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729793 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729796 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729800 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729804 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729807 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729810 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729813 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729816 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:32.798306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729819 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729821 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729824 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729827 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729829 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729832 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729835 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729837 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729840 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729843 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729846 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729848 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729851 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729854 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729856 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729859 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729861 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729864 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729866 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:32.799335 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729870 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:32.799860 ip-10-0-140-37 systemd[1]: Started Kubernetes Kubelet. Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729873 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729876 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729879 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729882 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729885 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729887 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729890 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729893 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729895 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729898 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729900 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729903 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729905 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729908 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729912 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729914 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729917 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729919 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729922 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:32.800290 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729924 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729928 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729932 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729934 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729938 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729941 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729944 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729947 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729949 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729952 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729954 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729957 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729959 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729962 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729964 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729967 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729970 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729973 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729975 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:32.911306 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729978 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729981 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729984 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729987 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729989 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729992 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729995 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.729997 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730001 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730003 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730006 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730008 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730010 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730013 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730469 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730477 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730481 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730484 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730488 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730491 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:32.926882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730494 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730496 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730499 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730502 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730505 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730508 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730512 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730515 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730518 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730526 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730529 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730532 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730535 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730538 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730541 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730544 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730547 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730549 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730552 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:32.927783 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730555 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730559 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730561 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730564 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730567 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730569 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730572 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730574 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730577 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730580 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730583 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730585 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730588 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730590 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730592 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730596 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730599 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730601 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730604 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730607 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:32.928491 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730609 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730612 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730616 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730618 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730621 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730624 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730627 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730629 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730631 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730634 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730636 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730639 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730642 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730645 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730647 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730650 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730652 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730655 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730658 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730660 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:32.929083 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730663 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730665 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730668 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730670 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730673 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730675 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730678 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730681 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730687 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730689 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730692 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730694 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730697 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730699 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730703 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730706 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730708 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730712 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730714 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730717 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:32.929745 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730719 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730722 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730726 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730729 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730732 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730734 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730737 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730739 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730742 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730745 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730747 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730750 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730752 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.730755 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730822 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730829 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730836 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730840 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730844 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730848 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730852 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 12:27:32.930360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730857 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730860 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730863 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730867 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730870 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730877 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730880 2584 flags.go:64] FLAG: --cgroup-root="" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730882 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730885 2584 flags.go:64] FLAG: --client-ca-file="" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730890 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730893 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730897 2584 flags.go:64] FLAG: --cluster-domain="" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730900 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730903 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730906 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730910 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730914 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730917 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730920 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730923 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730926 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730929 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730932 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730936 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 12:27:32.931004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730939 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730943 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730946 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730949 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730952 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730955 2584 flags.go:64] FLAG: --enable-server="true" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730957 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730963 2584 flags.go:64] FLAG: --event-burst="100" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730966 2584 flags.go:64] FLAG: --event-qps="50" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730969 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730972 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730980 2584 flags.go:64] FLAG: --eviction-hard="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730984 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730989 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730992 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730995 2584 flags.go:64] FLAG: --eviction-soft="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.730997 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731000 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731004 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731007 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731009 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731012 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731015 2584 flags.go:64] FLAG: --feature-gates="" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731019 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731022 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 12:27:32.931707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731025 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731028 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731031 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731034 2584 flags.go:64] FLAG: --help="false" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731037 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-140-37.ec2.internal" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731040 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731043 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731046 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731049 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731052 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731055 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731058 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731061 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731064 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731066 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731069 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731072 2584 flags.go:64] FLAG: --kube-reserved="" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731075 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731079 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731082 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731086 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731089 2584 flags.go:64] FLAG: --lock-file="" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731092 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731094 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 12:27:32.932577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731097 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731103 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731106 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731109 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731112 2584 flags.go:64] FLAG: --logging-format="text" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731115 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731118 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731121 2584 flags.go:64] FLAG: --manifest-url="" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731124 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731129 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731132 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731136 2584 flags.go:64] FLAG: --max-pods="110" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731139 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731142 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731145 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731148 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731151 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731154 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731157 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731164 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731167 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731170 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731173 2584 flags.go:64] FLAG: --pod-cidr="" Jul 21 12:27:32.933276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731176 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731182 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731185 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731190 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731193 2584 flags.go:64] FLAG: --port="10250" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731196 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731201 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04f0c451fa6b0b61e" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731204 2584 flags.go:64] FLAG: --qos-reserved="" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731207 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731226 2584 flags.go:64] FLAG: --register-node="true" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731229 2584 flags.go:64] FLAG: --register-with-taints="" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731233 2584 flags.go:64] FLAG: --registry-burst="10" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731237 2584 flags.go:64] FLAG: --registry-qps="5" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731240 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731243 2584 flags.go:64] FLAG: --reserved-memory="" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731247 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731250 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731253 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731256 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731259 2584 flags.go:64] FLAG: --runonce="false" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731262 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731265 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731268 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731271 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731274 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731277 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 12:27:32.933982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731280 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731283 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731286 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731289 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731291 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731294 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731297 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731300 2584 flags.go:64] FLAG: --system-cgroups="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731303 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731310 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731313 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731316 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731322 2584 flags.go:64] FLAG: --tls-min-version="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731325 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731327 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731330 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731333 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731336 2584 flags.go:64] FLAG: --v="2" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731340 2584 flags.go:64] FLAG: --version="false" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731345 2584 flags.go:64] FLAG: --vmodule="" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731349 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.731352 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731467 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731471 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:32.934720 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731475 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731477 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731481 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731483 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731486 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731488 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731491 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731493 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731496 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731498 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731501 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731503 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731506 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731508 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731511 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731514 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731516 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731520 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731524 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731526 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:32.935530 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731531 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731533 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731536 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731539 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731542 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731545 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731547 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731550 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731553 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731556 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731558 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731561 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731564 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731566 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731569 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731571 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731574 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731576 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731579 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731581 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:32.936141 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731584 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731586 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731589 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731591 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731594 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731597 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731599 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731602 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731604 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731609 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731611 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731614 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731617 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731620 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731622 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731625 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731628 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731631 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731633 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731636 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:32.936733 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731638 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731642 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731644 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731647 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731649 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731652 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731655 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731657 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731659 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731662 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731665 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731667 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731670 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731673 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731677 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731680 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731683 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731686 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731689 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:32.937285 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731691 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731694 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731698 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731702 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731705 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731709 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731712 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731714 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731717 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731719 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731722 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731725 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731727 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731730 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731732 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731737 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731741 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:32.937882 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.731744 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.732257 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739155 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739174 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739194 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739201 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739281 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739288 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739291 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739294 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739297 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739300 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739303 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739307 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739310 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:32.938376 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739313 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739316 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739319 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739322 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739324 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739327 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739330 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739333 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739335 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739338 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739341 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739344 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739347 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739349 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739351 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739354 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739357 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739360 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739363 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739365 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:32.938840 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739368 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739371 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739373 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739376 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739378 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739381 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739384 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739388 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739392 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739395 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739398 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739404 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739406 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739409 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739412 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739415 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739417 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739420 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739423 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:32.939878 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739425 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739428 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739430 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739433 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739435 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739439 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739443 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739445 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739448 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739452 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739455 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739458 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739460 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739463 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739466 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739469 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739471 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739474 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739476 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:32.940456 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739478 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739481 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739484 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739486 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739488 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739492 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739495 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739498 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739500 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739503 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739506 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739508 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739512 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739514 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739517 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739519 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739522 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739525 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739527 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739530 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:33.181099 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739532 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739535 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739539 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739542 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739546 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739549 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739551 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739554 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739558 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739561 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739563 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739566 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739571 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739693 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739698 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:27:33.182053 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739701 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739704 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739706 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739711 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739714 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739718 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739720 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739723 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739726 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739729 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739731 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739734 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739737 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739739 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739742 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739744 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739747 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739750 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739753 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:27:33.182556 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739756 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739759 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739762 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739764 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739767 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739769 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739772 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739775 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739778 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739780 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739783 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739785 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739788 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739790 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739793 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739795 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739802 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739805 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739808 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:27:33.183060 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739811 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739814 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739816 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739819 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739821 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739824 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739828 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739831 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739834 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739836 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739839 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739841 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739844 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739848 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739851 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739854 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739856 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739859 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739862 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:27:33.183557 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739864 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739867 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739870 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739873 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739875 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739877 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739880 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739882 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739885 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739887 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739891 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739894 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739897 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739900 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739902 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739905 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739907 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739910 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739912 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739915 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739918 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:27:33.184062 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739920 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739924 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739927 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739930 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739933 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739937 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739940 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739943 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739947 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739950 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739952 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739955 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739957 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739960 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739963 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739965 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739968 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739971 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:27:33.184601 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:32.739974 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.739979 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.740118 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.742190 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.743336 2584 server.go:1019] "Starting client certificate rotation" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.743441 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.743483 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.745470 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.748967 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.761375 2584 log.go:25] "Validated CRI v1 runtime API" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.763382 2584 log.go:25] "Validated CRI v1 image API" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.764176 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 12:27:33.185077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.768103 2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 822c982b-c32d-4850-b2bb-082c6fc3fd52:/dev/nvme0n1p4 e9e65940-4fb4-4d4b-b90b-1f877b5863d6:/dev/nvme0n1p3] Jul 21 12:27:33.185461 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.768119 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 12:27:33.185461 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.773002 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.773257 2584 manager.go:216] Machine: {Timestamp:2026-07-21 12:27:32.772068766 +0000 UTC m=+0.294796473 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3077731 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23b3b5ff452ba43f8c0eb9dd68d743 SystemUUID:ec23b3b5-ff45-2ba4-3f8c-0eb9dd68d743 BootID:eddc504c-fbae-41ae-ac7c-54f861a4c0ee Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8b:a6:12:0f:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8b:a6:12:0f:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:4d:9a:f6:43:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.773363 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.773457 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.775520 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.775545 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.776169 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.776179 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.776186 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.776799 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.777908 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780719 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780845 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780876 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780889 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780909 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780925 2584 kubelet.go:398] "Adding apiserver pod source" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.780947 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 12:27:33.185625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.781407 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.781423 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.783472 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.788012 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.788050 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789580 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789600 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789611 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789620 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789632 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789641 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789653 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789667 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789679 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.789688 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.790638 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.790649 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.792566 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.792567 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.795489 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.795520 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.798708 2584 server.go:1268] "Started kubelet" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.798810 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.798916 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.798984 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.800428 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.801032 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.806541 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.806511 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-37.ec2.internal.18c44dc32022e462 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-37.ec2.internal,UID:ip-10-0-140-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-37.ec2.internal,},FirstTimestamp:2026-07-21 12:27:32.798678114 +0000 UTC m=+0.321405820,LastTimestamp:2026-07-21 12:27:32.798678114 +0000 UTC m=+0.321405820,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-37.ec2.internal,}" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.810190 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.810189 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.810300 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811101 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811428 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811446 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811487 2584 factory.go:153] Registering CRI-O factory Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811507 2584 factory.go:223] Registration of the crio container factory successfully Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811585 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811597 2584 factory.go:55] Registering systemd factory Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811605 2584 factory.go:223] Registration of the systemd container factory successfully Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811629 2584 factory.go:103] Registering Raw factory Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811587 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 21 12:27:33.186149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811644 2584 manager.go:1196] Started watching for new ooms in manager Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.811645 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.812055 2584 manager.go:349] Starting recovery of all containers Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.816834 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.817343 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.817411 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.818836 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l7h4j" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.826194 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l7h4j" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.826414 2584 manager.go:354] Recovery completed Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.835539 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.837447 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.837474 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.837485 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838032 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838042 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838053 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838069 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838735 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838758 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.838767 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.839918 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-37.ec2.internal.18c44dc32272aa93 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-37.ec2.internal,UID:ip-10-0-140-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-37.ec2.internal,},FirstTimestamp:2026-07-21 12:27:32.837460627 +0000 UTC m=+0.360188333,LastTimestamp:2026-07-21 12:27:32.837460627 +0000 UTC m=+0.360188333,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-37.ec2.internal,}" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.840464 2584 policy_none.go:49] "None policy: Start" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.840478 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.840490 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.841950 2584 policy_none.go:47] "Start" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.896923 2584 manager.go:337] "Starting Device Plugin manager" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.896967 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.896978 2584 server.go:87] "Starting device plugin registration server" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.897269 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.897283 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.897563 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.897643 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.897653 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.898121 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.898177 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.956533 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.958099 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.958135 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.958163 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 12:27:33.187225 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:32.958197 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:32.960885 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.001146 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.002251 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.002281 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.002295 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.002322 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.009269 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.009289 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-37.ec2.internal\": node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.036310 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.058295 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal"] Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.058380 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.059225 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.059255 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.059272 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.062054 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.062083 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.062296 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064037 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064067 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064073 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064096 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064110 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.064080 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.065373 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.065393 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.066025 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.066051 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.066069 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.082743 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-37.ec2.internal\" not found" node="ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.086448 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-37.ec2.internal\" not found" node="ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.113171 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.113197 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.113243 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a870da13b5e904749212ecd217225c3b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"a870da13b5e904749212ecd217225c3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.188246 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.136390 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213765 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213800 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213815 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a870da13b5e904749212ecd217225c3b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"a870da13b5e904749212ecd217225c3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213866 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213873 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de6e2281e23a03a0cb31e85cf79937cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal\" (UID: \"de6e2281e23a03a0cb31e85cf79937cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.213913 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a870da13b5e904749212ecd217225c3b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-37.ec2.internal\" (UID: \"a870da13b5e904749212ecd217225c3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.236853 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.337318 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.385272 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.410685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.389236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.626586 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.437747 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.626586 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.538351 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.626586 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.600706 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.639061 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-37.ec2.internal\" not found" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.641308 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.711380 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.723268 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.724097 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.732108 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.743221 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.743338 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.743370 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.743380 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.781450 2584 apiserver.go:52] "Watching apiserver" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.800055 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.800770 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6k9r9","kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8","openshift-dns/node-resolver-r27b5","openshift-image-registry/node-ca-7rvj2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal","openshift-multus/multus-additional-cni-plugins-l5bqq","openshift-multus/multus-kv5qq","kube-system/global-pull-secret-syncer-xzw56","openshift-cluster-node-tuning-operator/tuned-vwp8p","openshift-multus/network-metrics-daemon-mwcl4","openshift-network-diagnostics/network-check-target-dl2nt","openshift-network-operator/iptables-alerter-4r27t","openshift-ovn-kubernetes/ovnkube-node-rlm2m"] Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.803130 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.803207 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.804822 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.804884 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.806096 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.807231 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808128 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808133 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808199 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808231 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b7dcf\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808259 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808274 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.808425 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.809231 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.809296 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wdqxt\"" Jul 21 12:27:33.819794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.809317 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.809368 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810255 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810279 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810296 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810305 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f869m\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810581 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.810623 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.811288 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.811338 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.811697 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g7qtq\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.811733 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.811742 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.812349 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l4wzx\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.812429 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.812828 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.814479 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.815142 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-774nl\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.815334 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.815490 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.816235 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.816817 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.816887 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.817004 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xmszp\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.817164 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.817294 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.818103 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.818391 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.818528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8tnml\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.819455 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.819651 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.819793 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.819924 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.820065 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.820494 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vqdx7\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.820664 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 12:27:33.820967 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.820832 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821596 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821623 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e335be90-a7bf-4077-a72a-694dbddcb799-host\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821640 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e335be90-a7bf-4077-a72a-694dbddcb799-serviceca\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821654 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-system-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821668 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-k8s-cni-cncf-io\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821682 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-kubelet\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821707 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-multus-daemon-config\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821735 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8b5a32b-9781-4981-bb66-3863b4ade135-hosts-file\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821767 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-netns\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821803 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-conf-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821836 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-etc-kubernetes\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.821903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821906 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-cnibin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821925 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c78sp\" (UniqueName: \"kubernetes.io/projected/a9c112e1-146f-4d04-81d2-a46117f84327-kube-api-access-c78sp\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821938 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-system-cni-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.821963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-os-release\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822060 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-bin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822077 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-hostroot\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-kubelet-config\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822106 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cnibin\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822147 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-multus\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-multus-certs\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822202 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-dbus\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822252 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10a8417d-5d86-4639-ba93-7de9d9bb9395-agent-certs\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.822409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822273 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-os-release\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822290 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjlv7\" (UniqueName: \"kubernetes.io/projected/cc969ba9-c0ec-4e44-8aeb-cbb820972411-kube-api-access-hjlv7\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822321 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpmsz\" (UniqueName: \"kubernetes.io/projected/e335be90-a7bf-4077-a72a-694dbddcb799-kube-api-access-bpmsz\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822342 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10a8417d-5d86-4639-ba93-7de9d9bb9395-konnectivity-ca\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822358 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822382 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-cni-binary-copy\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822396 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b5a32b-9781-4981-bb66-3863b4ade135-tmp-dir\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822410 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g62g\" (UniqueName: \"kubernetes.io/projected/c661bc0b-43a7-4602-9dc8-654b8095ce4a-kube-api-access-5g62g\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822424 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-socket-dir-parent\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.822950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.822437 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjbfv\" (UniqueName: \"kubernetes.io/projected/f8b5a32b-9781-4981-bb66-3863b4ade135-kube-api-access-qjbfv\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.825700 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.834279 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:22:32 +0000 UTC" deadline="2028-02-23 14:40:01.391690124 +0000 UTC" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.834311 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13970h12m27.557381092s" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.886266 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mhbh5" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.904135 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mhbh5" Jul 21 12:27:33.918950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.911610 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925788 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-netns\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925822 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-etc-kubernetes\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925846 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-device-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925874 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-etc-selinux\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925905 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-etc-kubernetes\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925920 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925905 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-netns\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925954 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.925982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4qgz\" (UniqueName: \"kubernetes.io/projected/9e98f882-eee7-47f3-a448-54d03048b868-kube-api-access-s4qgz\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926013 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-os-release\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.926025 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926050 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.926145 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:27:34.42612347 +0000 UTC m=+1.948851184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:33.994892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-bin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926251 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-kubelet-config\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cnibin\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926317 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-multus-certs\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926338 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd23d57-57d9-4c27-a66f-a4524f74fd75-iptables-alerter-script\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926360 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-var-lib-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-etc-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-dbus\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10a8417d-5d86-4639-ba93-7de9d9bb9395-agent-certs\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926464 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926485 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-conf\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926507 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-run\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926526 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-registration-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjlv7\" (UniqueName: \"kubernetes.io/projected/cc969ba9-c0ec-4e44-8aeb-cbb820972411-kube-api-access-hjlv7\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.995707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926567 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-kubelet\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926592 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-node-log\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926645 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-log-socket\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926667 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-env-overrides\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926687 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-sys-fs\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g62g\" (UniqueName: \"kubernetes.io/projected/c661bc0b-43a7-4602-9dc8-654b8095ce4a-kube-api-access-5g62g\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926731 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-socket-dir-parent\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926799 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-systemd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926819 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926844 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysconfig\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926863 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-systemd\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzp2k\" (UniqueName: \"kubernetes.io/projected/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kube-api-access-hzp2k\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-dbus\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926906 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.926944 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-os-release\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927056 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e335be90-a7bf-4077-a72a-694dbddcb799-host\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927081 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-system-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927103 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-k8s-cni-cncf-io\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-kubelet\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927166 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8b5a32b-9781-4981-bb66-3863b4ade135-hosts-file\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927187 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-slash\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927228 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-config\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927251 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-conf-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927274 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-systemd-units\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927294 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovn-node-metrics-cert\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927314 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-cnibin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c78sp\" (UniqueName: \"kubernetes.io/projected/a9c112e1-146f-4d04-81d2-a46117f84327-kube-api-access-c78sp\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-netns\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927390 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfph\" (UniqueName: \"kubernetes.io/projected/e2b79acb-4d11-4a73-bfc6-58811cdb8850-kube-api-access-ptfph\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927420 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-lib-modules\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927441 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-var-lib-kubelet\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-tmp\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-system-cni-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-hostroot\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.996955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-multus\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927578 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-script-lib\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927620 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-kubernetes\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-sys\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927664 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-socket-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927712 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-multus-certs\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927742 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-bin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.927782 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-kubelet-config\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928024 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928030 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cnibin\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928366 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-socket-dir-parent\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928403 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928817 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e335be90-a7bf-4077-a72a-694dbddcb799-host\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928878 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-system-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928923 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-run-k8s-cni-cncf-io\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.928965 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-kubelet\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929012 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8b5a32b-9781-4981-bb66-3863b4ade135-hosts-file\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.997629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929070 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-conf-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929142 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-cnibin\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929169 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929181 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929182 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc969ba9-c0ec-4e44-8aeb-cbb820972411-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929280 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc969ba9-c0ec-4e44-8aeb-cbb820972411-system-cni-dir\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929291 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-hostroot\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-os-release\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.929344 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929363 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-bin\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929399 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-host-var-lib-cni-multus\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.929458 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:27:34.429423239 +0000 UTC m=+1.952150947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929481 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmsz\" (UniqueName: \"kubernetes.io/projected/e335be90-a7bf-4077-a72a-694dbddcb799-kube-api-access-bpmsz\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929499 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-os-release\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929541 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10a8417d-5d86-4639-ba93-7de9d9bb9395-konnectivity-ca\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929652 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929732 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-cni-binary-copy\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.998631 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929767 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9c112e1-146f-4d04-81d2-a46117f84327-multus-cni-dir\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b5a32b-9781-4981-bb66-3863b4ade135-tmp-dir\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929900 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd23d57-57d9-4c27-a66f-a4524f74fd75-host-slash\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-etc-tuned\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.929964 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjbfv\" (UniqueName: \"kubernetes.io/projected/f8b5a32b-9781-4981-bb66-3863b4ade135-kube-api-access-qjbfv\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930004 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq747\" (UniqueName: \"kubernetes.io/projected/1cd23d57-57d9-4c27-a66f-a4524f74fd75-kube-api-access-fq747\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930099 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10a8417d-5d86-4639-ba93-7de9d9bb9395-konnectivity-ca\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930135 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b5a32b-9781-4981-bb66-3863b4ade135-tmp-dir\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930140 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-modprobe-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930173 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e335be90-a7bf-4077-a72a-694dbddcb799-serviceca\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-multus-daemon-config\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930261 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-ovn\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930281 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-netd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930302 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930323 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-host\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930437 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-cni-binary-copy\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930577 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9c112e1-146f-4d04-81d2-a46117f84327-multus-daemon-config\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.999161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.930588 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e335be90-a7bf-4077-a72a-694dbddcb799-serviceca\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.932027 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10a8417d-5d86-4639-ba93-7de9d9bb9395-agent-certs\") pod \"konnectivity-agent-6k9r9\" (UID: \"10a8417d-5d86-4639-ba93-7de9d9bb9395\") " pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.936103 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.936124 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.936140 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:33.936180 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:34.436168521 +0000 UTC m=+1.958896224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.940317 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjlv7\" (UniqueName: \"kubernetes.io/projected/cc969ba9-c0ec-4e44-8aeb-cbb820972411-kube-api-access-hjlv7\") pod \"multus-additional-cni-plugins-l5bqq\" (UID: \"cc969ba9-c0ec-4e44-8aeb-cbb820972411\") " pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.940423 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g62g\" (UniqueName: \"kubernetes.io/projected/c661bc0b-43a7-4602-9dc8-654b8095ce4a-kube-api-access-5g62g\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.940459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c78sp\" (UniqueName: \"kubernetes.io/projected/a9c112e1-146f-4d04-81d2-a46117f84327-kube-api-access-c78sp\") pod \"multus-kv5qq\" (UID: \"a9c112e1-146f-4d04-81d2-a46117f84327\") " pod="openshift-multus/multus-kv5qq" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.940635 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmsz\" (UniqueName: \"kubernetes.io/projected/e335be90-a7bf-4077-a72a-694dbddcb799-kube-api-access-bpmsz\") pod \"node-ca-7rvj2\" (UID: \"e335be90-a7bf-4077-a72a-694dbddcb799\") " pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.941001 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjbfv\" (UniqueName: \"kubernetes.io/projected/f8b5a32b-9781-4981-bb66-3863b4ade135-kube-api-access-qjbfv\") pod \"node-resolver-r27b5\" (UID: \"f8b5a32b-9781-4981-bb66-3863b4ade135\") " pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:33.999756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:33.954017 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kv5qq" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030751 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-bin\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd23d57-57d9-4c27-a66f-a4524f74fd75-host-slash\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-bin\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030866 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd23d57-57d9-4c27-a66f-a4524f74fd75-host-slash\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030895 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-etc-tuned\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030920 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq747\" (UniqueName: \"kubernetes.io/projected/1cd23d57-57d9-4c27-a66f-a4524f74fd75-kube-api-access-fq747\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-modprobe-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030971 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-ovn\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.030993 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-netd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031014 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031039 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-host\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-device-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-etc-selinux\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031098 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-modprobe-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031116 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031137 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4qgz\" (UniqueName: \"kubernetes.io/projected/9e98f882-eee7-47f3-a448-54d03048b868-kube-api-access-s4qgz\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072226 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031158 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-ovn\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031181 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-cni-netd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031136 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031253 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031256 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-device-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031332 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd23d57-57d9-4c27-a66f-a4524f74fd75-iptables-alerter-script\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031399 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-var-lib-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031417 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-etc-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031462 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-etc-selinux\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031469 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-conf\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-host\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-run\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031553 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-registration-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031566 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-d\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.072970 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031579 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-kubelet\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031605 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-node-log\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031620 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-run\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031628 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysctl-conf\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031668 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-kubelet\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-registration-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031674 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-node-log\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031628 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-log-socket\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031700 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-var-lib-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031705 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-log-socket\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031711 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-env-overrides\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031720 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-etc-openvswitch\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031736 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-sys-fs\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031760 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-systemd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysconfig\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-systemd\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.073848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031836 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-sys-fs\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031855 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzp2k\" (UniqueName: \"kubernetes.io/projected/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kube-api-access-hzp2k\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031875 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031900 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-run-systemd\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031905 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-slash\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031939 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-slash\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031946 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-sysconfig\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031939 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-config\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-systemd-units\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.031989 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-systemd\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovn-node-metrics-cert\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-netns\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032058 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfph\" (UniqueName: \"kubernetes.io/projected/e2b79acb-4d11-4a73-bfc6-58811cdb8850-kube-api-access-ptfph\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-lib-modules\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032099 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-var-lib-kubelet\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032110 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-env-overrides\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032119 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-tmp\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.074497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-script-lib\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032188 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-kubernetes\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032231 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-sys\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032238 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-systemd-units\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032247 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-socket-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032354 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-socket-dir\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032474 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-config\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032189 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2b79acb-4d11-4a73-bfc6-58811cdb8850-host-run-netns\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032500 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd23d57-57d9-4c27-a66f-a4524f74fd75-iptables-alerter-script\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032548 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-etc-kubernetes\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032598 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-sys\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-lib-modules\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032642 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e98f882-eee7-47f3-a448-54d03048b868-var-lib-kubelet\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.032811 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovnkube-script-lib\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.033795 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-etc-tuned\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.034342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e98f882-eee7-47f3-a448-54d03048b868-tmp\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.034523 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2b79acb-4d11-4a73-bfc6-58811cdb8850-ovn-node-metrics-cert\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.045818 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4qgz\" (UniqueName: \"kubernetes.io/projected/9e98f882-eee7-47f3-a448-54d03048b868-kube-api-access-s4qgz\") pod \"tuned-vwp8p\" (UID: \"9e98f882-eee7-47f3-a448-54d03048b868\") " pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.075038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.048712 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq747\" (UniqueName: \"kubernetes.io/projected/1cd23d57-57d9-4c27-a66f-a4524f74fd75-kube-api-access-fq747\") pod \"iptables-alerter-4r27t\" (UID: \"1cd23d57-57d9-4c27-a66f-a4524f74fd75\") " pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.075675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.048949 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfph\" (UniqueName: \"kubernetes.io/projected/e2b79acb-4d11-4a73-bfc6-58811cdb8850-kube-api-access-ptfph\") pod \"ovnkube-node-rlm2m\" (UID: \"e2b79acb-4d11-4a73-bfc6-58811cdb8850\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.075675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.049331 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzp2k\" (UniqueName: \"kubernetes.io/projected/be398ff0-a6fc-4c29-ab5e-f001e9be5e69-kube-api-access-hzp2k\") pod \"aws-ebs-csi-driver-node-jznz8\" (UID: \"be398ff0-a6fc-4c29-ab5e-f001e9be5e69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.075675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.051309 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4r27t" Jul 21 12:27:34.075675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.056245 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.101983 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda870da13b5e904749212ecd217225c3b.slice/crio-2c5c400437a248b72da520a018445080676262606cc7ab237a5aedf66ed98597 WatchSource:0}: Error finding container 2c5c400437a248b72da520a018445080676262606cc7ab237a5aedf66ed98597: Status 404 returned error can't find the container with id 2c5c400437a248b72da520a018445080676262606cc7ab237a5aedf66ed98597 Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.102296 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6e2281e23a03a0cb31e85cf79937cb.slice/crio-b30cedbe21ffce0612c45610ab64160e1195e56610c09e008976e4455394193c WatchSource:0}: Error finding container b30cedbe21ffce0612c45610ab64160e1195e56610c09e008976e4455394193c: Status 404 returned error can't find the container with id b30cedbe21ffce0612c45610ab64160e1195e56610c09e008976e4455394193c Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.109872 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.147437 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.155666 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc969ba9_c0ec_4e44_8aeb_cbb820972411.slice/crio-be1cefef02bc79b55c07b855f167d58725bdee9f2a5c665f9643ef4787545524 WatchSource:0}: Error finding container be1cefef02bc79b55c07b855f167d58725bdee9f2a5c665f9643ef4787545524: Status 404 returned error can't find the container with id be1cefef02bc79b55c07b855f167d58725bdee9f2a5c665f9643ef4787545524 Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.173809 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r27b5" Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.180666 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b5a32b_9781_4981_bb66_3863b4ade135.slice/crio-8a7e8817c3c7030488fd64940c392ed38656f1c0e2d6c4d8d26ff4f6006a828c WatchSource:0}: Error finding container 8a7e8817c3c7030488fd64940c392ed38656f1c0e2d6c4d8d26ff4f6006a828c: Status 404 returned error can't find the container with id 8a7e8817c3c7030488fd64940c392ed38656f1c0e2d6c4d8d26ff4f6006a828c Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.196833 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7rvj2" Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.205744 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode335be90_a7bf_4077_a72a_694dbddcb799.slice/crio-fc9fcf6e942769555bfb39df4775daf69817fea072e80eca0f75d59a6fcb0520 WatchSource:0}: Error finding container fc9fcf6e942769555bfb39df4775daf69817fea072e80eca0f75d59a6fcb0520: Status 404 returned error can't find the container with id fc9fcf6e942769555bfb39df4775daf69817fea072e80eca0f75d59a6fcb0520 Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.229671 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:34.275331 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.238255 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a8417d_5d86_4639_ba93_7de9d9bb9395.slice/crio-22dd34601aad0ee4454075907eb72af5262711323b3fc327761a5434155988e9 WatchSource:0}: Error finding container 22dd34601aad0ee4454075907eb72af5262711323b3fc327761a5434155988e9: Status 404 returned error can't find the container with id 22dd34601aad0ee4454075907eb72af5262711323b3fc327761a5434155988e9 Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.282682 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.293546 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e98f882_eee7_47f3_a448_54d03048b868.slice/crio-b22cc9ff02504f7a07635f3f375260281fece40b38a4d1bb979cd9eebb8ffa9a WatchSource:0}: Error finding container b22cc9ff02504f7a07635f3f375260281fece40b38a4d1bb979cd9eebb8ffa9a: Status 404 returned error can't find the container with id b22cc9ff02504f7a07635f3f375260281fece40b38a4d1bb979cd9eebb8ffa9a Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.307557 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.314807 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c112e1_146f_4d04_81d2_a46117f84327.slice/crio-9d734beb2bcfd72ce05b203876d33242715f206b9a75b9179bf161af64fa0dda WatchSource:0}: Error finding container 9d734beb2bcfd72ce05b203876d33242715f206b9a75b9179bf161af64fa0dda: Status 404 returned error can't find the container with id 9d734beb2bcfd72ce05b203876d33242715f206b9a75b9179bf161af64fa0dda Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.317785 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe398ff0_a6fc_4c29_ab5e_f001e9be5e69.slice/crio-77066141d0dd11e1f5f8e428b9023d8e2be9bad7c34ec6c455c3ae8bffd2ca42 WatchSource:0}: Error finding container 77066141d0dd11e1f5f8e428b9023d8e2be9bad7c34ec6c455c3ae8bffd2ca42: Status 404 returned error can't find the container with id 77066141d0dd11e1f5f8e428b9023d8e2be9bad7c34ec6c455c3ae8bffd2ca42 Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:27:34.373066 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd23d57_57d9_4c27_a66f_a4524f74fd75.slice/crio-abd69ea59f89012038d8add67d27962064c5ad6271b3aa276ed3a4eb66370593 WatchSource:0}: Error finding container abd69ea59f89012038d8add67d27962064c5ad6271b3aa276ed3a4eb66370593: Status 404 returned error can't find the container with id abd69ea59f89012038d8add67d27962064c5ad6271b3aa276ed3a4eb66370593 Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.396039 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.435658 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.435702 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.435803 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.435807 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.435857 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:27:35.435843941 +0000 UTC m=+2.958571635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:34.498977 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.435871 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:27:35.435864927 +0000 UTC m=+2.958592620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:34.683023 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.536293 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:34.683023 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.536442 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:34.683023 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.536458 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:34.683023 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.536467 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:34.683023 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.536522 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:35.536506796 +0000 UTC m=+3.059234491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:34.914679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.900937 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:34.914679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.906000 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:22:33 +0000 UTC" deadline="2028-02-29 08:11:10.955306098 +0000 UTC" Jul 21 12:27:34.914679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.906063 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14107h43m36.049247999s" Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.961861 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:34.962009 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.970588 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kv5qq" event={"ID":"a9c112e1-146f-4d04-81d2-a46117f84327","Type":"ContainerStarted","Data":"9d734beb2bcfd72ce05b203876d33242715f206b9a75b9179bf161af64fa0dda"} Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.974416 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" event={"ID":"9e98f882-eee7-47f3-a448-54d03048b868","Type":"ContainerStarted","Data":"b22cc9ff02504f7a07635f3f375260281fece40b38a4d1bb979cd9eebb8ffa9a"} Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.977077 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6k9r9" event={"ID":"10a8417d-5d86-4639-ba93-7de9d9bb9395","Type":"ContainerStarted","Data":"22dd34601aad0ee4454075907eb72af5262711323b3fc327761a5434155988e9"} Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.979990 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7rvj2" event={"ID":"e335be90-a7bf-4077-a72a-694dbddcb799","Type":"ContainerStarted","Data":"fc9fcf6e942769555bfb39df4775daf69817fea072e80eca0f75d59a6fcb0520"} Jul 21 12:27:34.983729 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.981714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r27b5" event={"ID":"f8b5a32b-9781-4981-bb66-3863b4ade135","Type":"ContainerStarted","Data":"8a7e8817c3c7030488fd64940c392ed38656f1c0e2d6c4d8d26ff4f6006a828c"} Jul 21 12:27:34.987654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.986777 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"de6e2281e23a03a0cb31e85cf79937cb","Type":"ContainerStarted","Data":"b30cedbe21ffce0612c45610ab64160e1195e56610c09e008976e4455394193c"} Jul 21 12:27:34.989485 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.988655 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerStarted","Data":"be1cefef02bc79b55c07b855f167d58725bdee9f2a5c665f9643ef4787545524"} Jul 21 12:27:34.995711 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.993226 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" event={"ID":"a870da13b5e904749212ecd217225c3b","Type":"ContainerStarted","Data":"2c5c400437a248b72da520a018445080676262606cc7ab237a5aedf66ed98597"} Jul 21 12:27:34.995975 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.995949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4r27t" event={"ID":"1cd23d57-57d9-4c27-a66f-a4524f74fd75","Type":"ContainerStarted","Data":"abd69ea59f89012038d8add67d27962064c5ad6271b3aa276ed3a4eb66370593"} Jul 21 12:27:34.999561 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:34.999413 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" event={"ID":"be398ff0-a6fc-4c29-ab5e-f001e9be5e69","Type":"ContainerStarted","Data":"77066141d0dd11e1f5f8e428b9023d8e2be9bad7c34ec6c455c3ae8bffd2ca42"} Jul 21 12:27:35.444072 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.443974 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:35.444072 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.444071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:35.444334 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.444200 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:35.444334 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.444279 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:27:37.444260467 +0000 UTC m=+4.966988164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:35.444707 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.444685 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:35.444811 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.444746 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:27:37.444728936 +0000 UTC m=+4.967456630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:35.544429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.544386 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:35.544620 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.544604 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:35.544709 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.544626 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:35.544709 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.544639 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:35.544709 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.544698 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:37.54467909 +0000 UTC m=+5.067406800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:35.906749 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.906612 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:22:33 +0000 UTC" deadline="2028-02-17 10:27:25.894683296 +0000 UTC" Jul 21 12:27:35.906749 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.906648 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13821h59m49.988040016s" Jul 21 12:27:35.959916 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.959121 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:35.959916 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.959267 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:35.959916 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:35.959707 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:35.959916 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:35.959819 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:36.010764 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:36.010682 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"9a3b443d15b227b583d251bce568f4a849cde6aa76ff5bc2ddc2334550b2bf6c"} Jul 21 12:27:36.016447 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:36.015654 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:36.283778 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:36.283685 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:27:36.960887 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:36.960851 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:36.961541 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:36.961023 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:37.460394 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.460521 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:37.460526 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.460603 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:27:41.46057985 +0000 UTC m=+8.983307556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.460616 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:37.460732 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.460673 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:27:41.460654455 +0000 UTC m=+8.983382151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:37.561978 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:37.561324 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:37.561978 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.561496 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:37.561978 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.561519 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:37.561978 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.561537 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:37.561978 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.561596 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:41.561578139 +0000 UTC m=+9.084305838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:37.958689 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:37.958606 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:37.958845 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.958743 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:37.959192 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:37.959170 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:37.959323 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:37.959290 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:38.959608 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:38.958720 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:38.959608 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:38.958858 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:39.959060 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:39.959021 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:39.959300 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:39.959167 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:39.959366 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:39.959297 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:39.959423 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:39.959376 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:40.962416 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:40.962387 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:40.962815 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:40.962536 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:41.499300 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:41.499247 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:41.499506 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:41.499356 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:41.499506 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.499412 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:41.499506 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.499445 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:41.499506 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.499480 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:27:49.499461314 +0000 UTC m=+17.022189022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:41.499506 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.499500 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:27:49.499490333 +0000 UTC m=+17.022218027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:41.601193 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:41.600484 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:41.601193 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.600705 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:41.601193 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.600735 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:41.601193 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.600750 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:41.601193 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.600819 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:49.600797539 +0000 UTC m=+17.123525246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:41.959399 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:41.959272 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:41.959662 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.959432 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:41.959758 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:41.959746 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:41.959853 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:41.959833 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:42.958982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:42.958945 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:42.959448 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:42.959121 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:43.959177 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:43.959134 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:43.959644 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:43.959134 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:43.959644 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:43.959292 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:43.959644 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:43.959304 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:44.959611 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:44.959573 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:44.960056 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:44.959746 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:45.958405 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:45.958366 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:45.958588 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:45.958367 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:45.958588 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:45.958503 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:45.958716 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:45.958592 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:46.958382 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:46.958342 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:46.958862 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:46.958491 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:47.959044 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:47.959000 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:47.959509 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:47.959053 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:47.959509 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:47.959150 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:47.959509 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:47.959287 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:48.961275 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:48.961240 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:48.961646 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:48.961385 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:49.564200 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:49.564158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:49.564450 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:49.564272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:49.564450 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.564338 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:49.564450 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.564354 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:49.564450 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.564410 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:28:05.564393129 +0000 UTC m=+33.087120824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:27:49.564450 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.564429 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:05.564419799 +0000 UTC m=+33.087147493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:27:49.664870 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:49.664777 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:49.665057 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.664972 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:27:49.665057 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.664998 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:27:49.665057 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.665011 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:49.665249 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.665080 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:05.665060376 +0000 UTC m=+33.187788087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:27:49.959286 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:49.959249 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:49.959523 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:49.959250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:49.959523 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.959369 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:49.959523 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:49.959464 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:50.958997 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:50.958957 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:50.959441 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:50.959118 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:51.959256 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:51.959201 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:51.959707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:51.959201 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:51.959707 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:51.959363 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:51.959707 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:51.959468 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:52.960031 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:52.959978 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:52.960514 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:52.960091 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:53.960336 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:53.960104 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:53.960813 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:53.960469 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:53.960813 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:53.960118 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:53.960813 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:53.960652 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:54.050397 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.050362 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" event={"ID":"a870da13b5e904749212ecd217225c3b","Type":"ContainerStarted","Data":"5179e3591ec011c1528ffa50f7daed0a1e62fd76927a3bfe7e0879444d89af28"} Jul 21 12:27:54.051870 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.051846 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kv5qq" event={"ID":"a9c112e1-146f-4d04-81d2-a46117f84327","Type":"ContainerStarted","Data":"edb10836ec5ff197695e98233145f32344a5397bc4c1298f98ff583b5d8cb8d8"} Jul 21 12:27:54.053315 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.053286 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" event={"ID":"9e98f882-eee7-47f3-a448-54d03048b868","Type":"ContainerStarted","Data":"4d18f0c1b5cd2f36dcc3605af8f33840fbb1c8c0137b559f7efc8259df23b499"} Jul 21 12:27:54.065078 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.063039 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-37.ec2.internal" podStartSLOduration=21.063019298 podStartE2EDuration="21.063019298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:54.062827286 +0000 UTC m=+21.585555003" watchObservedRunningTime="2026-07-21 12:27:54.063019298 +0000 UTC m=+21.585747016" Jul 21 12:27:54.078767 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.078718 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kv5qq" podStartSLOduration=1.673812936 podStartE2EDuration="21.078702651s" totalImagesPullingTime="19.404889715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:54.078353073 +0000 UTC m=+21.601080793" watchObservedRunningTime="2026-07-21 12:27:54.078702651 +0000 UTC m=+21.601430367" Jul 21 12:27:54.959786 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:54.959510 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:54.959930 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:54.959903 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:55.056991 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.056958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" event={"ID":"be398ff0-a6fc-4c29-ab5e-f001e9be5e69","Type":"ContainerStarted","Data":"4a090a26e58cfca66c3493505a7e3fe8d57d46f1c4aece34bb3a731ecb12f264"} Jul 21 12:27:55.058552 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.058363 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6k9r9" event={"ID":"10a8417d-5d86-4639-ba93-7de9d9bb9395","Type":"ContainerStarted","Data":"2cbb6574f43c68146195c16ca611eb49c9b20aa433cbb81bd4058931b60e44fe"} Jul 21 12:27:55.059793 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.059768 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7rvj2" event={"ID":"e335be90-a7bf-4077-a72a-694dbddcb799","Type":"ContainerStarted","Data":"263a800f4acc2263abb955395fd9b8c754c40a7e4c16e25e86b0a49a0b19e772"} Jul 21 12:27:55.061142 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.061120 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r27b5" event={"ID":"f8b5a32b-9781-4981-bb66-3863b4ade135","Type":"ContainerStarted","Data":"4d66b934009eb52a4579ff2d504c3aa4650b313decfa1e1f05d254787166751b"} Jul 21 12:27:55.062615 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.062593 2584 generic.go:356] "Generic (PLEG): container finished" podID="de6e2281e23a03a0cb31e85cf79937cb" containerID="ceb23f94d4b7d854901db8d3ed146c0153c951bae4a1510ef3c89e40032b9889" exitCode=0 Jul 21 12:27:55.062704 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.062664 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"de6e2281e23a03a0cb31e85cf79937cb","Type":"ContainerDied","Data":"ceb23f94d4b7d854901db8d3ed146c0153c951bae4a1510ef3c89e40032b9889"} Jul 21 12:27:55.064121 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.064099 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="d988002ff942a3d4b1bb4cd03f3c2161e35e826ef0291ce1543193fa642461fd" exitCode=0 Jul 21 12:27:55.064203 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.064178 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"d988002ff942a3d4b1bb4cd03f3c2161e35e826ef0291ce1543193fa642461fd"} Jul 21 12:27:55.069371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069350 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:27:55.069696 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069675 2584 generic.go:356] "Generic (PLEG): container finished" podID="e2b79acb-4d11-4a73-bfc6-58811cdb8850" containerID="7b3894e208d4278c9e0043bb5205fea37aa45d27c2686aaf154bb8bf46b5dc00" exitCode=1 Jul 21 12:27:55.069804 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"b84b9b491619cb142f899287f36137b4c589fd543c807eec652978edc3c82b44"} Jul 21 12:27:55.069916 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"caa7b0d17299499b3436cc981a11decd8106568a3918f5aab7f320776d249b87"} Jul 21 12:27:55.069961 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069951 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"38b5a3699e6a5c7333694ff5664e3ebc0b98788bd56cf74d54f892df5604dc9a"} Jul 21 12:27:55.069994 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.069984 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerDied","Data":"7b3894e208d4278c9e0043bb5205fea37aa45d27c2686aaf154bb8bf46b5dc00"} Jul 21 12:27:55.070033 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.070023 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"bfbb60193a1ade1ddf9bb5715556409b2b7cd7580b8de8b447bd452cf81a47ce"} Jul 21 12:27:55.073201 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.073161 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vwp8p" podStartSLOduration=2.647659444 podStartE2EDuration="22.073142754s" totalImagesPullingTime="19.42548331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:54.093498382 +0000 UTC m=+21.616226110" watchObservedRunningTime="2026-07-21 12:27:55.073142754 +0000 UTC m=+22.595870466" Jul 21 12:27:55.073648 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.073622 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6k9r9" podStartSLOduration=2.612601794 podStartE2EDuration="22.073610405s" totalImagesPullingTime="19.461008611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:55.073143912 +0000 UTC m=+22.595871628" watchObservedRunningTime="2026-07-21 12:27:55.073610405 +0000 UTC m=+22.596338120" Jul 21 12:27:55.107790 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.107741 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r27b5" podStartSLOduration=2.562211499 podStartE2EDuration="22.107726461s" totalImagesPullingTime="19.545514962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:55.107402286 +0000 UTC m=+22.630130001" watchObservedRunningTime="2026-07-21 12:27:55.107726461 +0000 UTC m=+22.630454176" Jul 21 12:27:55.120985 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.120938 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7rvj2" podStartSLOduration=2.627305642 podStartE2EDuration="22.120924269s" totalImagesPullingTime="19.493618627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:55.120563783 +0000 UTC m=+22.643291499" watchObservedRunningTime="2026-07-21 12:27:55.120924269 +0000 UTC m=+22.643651984" Jul 21 12:27:55.959020 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.958980 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:55.959020 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:55.959013 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:55.959397 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:55.959096 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:55.959397 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:55.959165 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:56.075243 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.074923 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4r27t" event={"ID":"1cd23d57-57d9-4c27-a66f-a4524f74fd75","Type":"ContainerStarted","Data":"436eb26f147766cc4eee3fb631337af6903c2200b3e6488eb967cda6f1dfea7f"} Jul 21 12:27:56.078572 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.078544 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:27:56.079613 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.079587 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"72de5eb922058d7de3c7aeff812d65cc9f5c9d077cb4730f737fc09876c688cd"} Jul 21 12:27:56.092637 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.092579 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4r27t" podStartSLOduration=3.745668774 podStartE2EDuration="23.092554749s" totalImagesPullingTime="19.346885975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:56.092119283 +0000 UTC m=+23.614847000" watchObservedRunningTime="2026-07-21 12:27:56.092554749 +0000 UTC m=+23.615282468" Jul 21 12:27:56.426145 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.426119 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 12:27:56.909067 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.908957 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:27:56.426139105Z","UUID":"28cdb440-d0db-451c-8240-0d859d169349","Handler":null,"Name":"","Endpoint":""} Jul 21 12:27:56.911137 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.911105 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 12:27:56.911300 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.911150 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 12:27:56.961993 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:56.961959 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:56.962162 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:56.962078 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:57.084093 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:57.084045 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" event={"ID":"be398ff0-a6fc-4c29-ab5e-f001e9be5e69","Type":"ContainerStarted","Data":"ccde249206361cb642894359643a1e42cb847411dc79e4a5016a8ab3c76e5efb"} Jul 21 12:27:57.086144 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:57.086110 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" event={"ID":"de6e2281e23a03a0cb31e85cf79937cb","Type":"ContainerStarted","Data":"d82d5fa4436f5792a816f3f24fa02cabbc8c15d71e79d38c232e0f38f7b9d709"} Jul 21 12:27:57.102254 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:57.102183 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-37.ec2.internal" podStartSLOduration=24.102162044 podStartE2EDuration="24.102162044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:57.101931381 +0000 UTC m=+24.624659098" watchObservedRunningTime="2026-07-21 12:27:57.102162044 +0000 UTC m=+24.624889761" Jul 21 12:27:57.959509 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:57.959413 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:57.959667 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:57.959413 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:57.959667 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:57.959532 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:57.959667 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:57.959614 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:27:58.090940 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:58.090888 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" event={"ID":"be398ff0-a6fc-4c29-ab5e-f001e9be5e69","Type":"ContainerStarted","Data":"1148648b40912dae1e0cb71f3e9dac0d25c2755c40339e81b3e954c5ba9bc350"} Jul 21 12:27:58.095048 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:58.095026 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:27:58.095496 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:58.095468 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"3762f72e01152e0aa54efddbad26ae943ab7a43d424cb6cb322ed899c16d3e4a"} Jul 21 12:27:58.109431 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:58.109377 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jznz8" podStartSLOduration=3.104233069 podStartE2EDuration="25.109357386s" totalImagesPullingTime="22.005124317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:58.108870113 +0000 UTC m=+25.631597830" watchObservedRunningTime="2026-07-21 12:27:58.109357386 +0000 UTC m=+25.632085102" Jul 21 12:27:58.962420 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:58.962389 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:27:58.962597 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:58.962539 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:27:59.853968 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:59.853920 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:59.854805 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:59.854618 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:27:59.959276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:59.959240 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:27:59.959453 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:27:59.959297 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:27:59.959453 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:59.959391 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:27:59.959639 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:27:59.959574 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:28:00.101095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:00.101058 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:28:00.101580 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:00.101554 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6k9r9" Jul 21 12:28:00.958789 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:00.958746 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:00.959481 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:00.958911 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:28:01.104332 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.104300 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="a4b15b159b437f8710e770b4de56176cb04da06b18db59346ee1a60ae32be118" exitCode=0 Jul 21 12:28:01.104513 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.104402 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"a4b15b159b437f8710e770b4de56176cb04da06b18db59346ee1a60ae32be118"} Jul 21 12:28:01.108015 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.107995 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:28:01.108431 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.108410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"345a6962e4ad8b03c7a1673616f00bddc3b5256d88d9b39db39a745480bb7474"} Jul 21 12:28:01.108921 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.108904 2584 scope.go:117] "RemoveContainer" containerID="7b3894e208d4278c9e0043bb5205fea37aa45d27c2686aaf154bb8bf46b5dc00" Jul 21 12:28:01.958702 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.958671 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:01.958891 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:01.958671 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:01.958891 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:01.958791 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:28:01.958891 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:01.958880 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:28:02.120016 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.119926 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:28:02.120927 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.120825 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" event={"ID":"e2b79acb-4d11-4a73-bfc6-58811cdb8850","Type":"ContainerStarted","Data":"05887c8c418cba070298b1fb30fe1e8c301617aa024966f9c6b9e38d03bf4263"} Jul 21 12:28:02.121263 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.121240 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:02.121396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.121322 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:02.121396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.121370 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:02.141847 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.141813 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:02.142398 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.142378 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:02.153496 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.153444 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" podStartSLOduration=9.863435191 podStartE2EDuration="29.153420771s" totalImagesPullingTime="19.28998558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:02.152068195 +0000 UTC m=+29.674795910" watchObservedRunningTime="2026-07-21 12:28:02.153420771 +0000 UTC m=+29.676148488" Jul 21 12:28:02.502535 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.502451 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzw56"] Jul 21 12:28:02.502685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.502567 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:02.502685 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:02.502649 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:28:02.505521 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.505493 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mwcl4"] Jul 21 12:28:02.505694 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.505632 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:02.505775 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:02.505752 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:28:02.506121 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.506095 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dl2nt"] Jul 21 12:28:02.506253 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:02.506186 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:02.506320 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:02.506288 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:28:03.125110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:03.125072 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="5888444284caa6496d688eca95543e2be652cd952e17ebb66ce5a29195cdfd8f" exitCode=0 Jul 21 12:28:03.125542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:03.125150 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"5888444284caa6496d688eca95543e2be652cd952e17ebb66ce5a29195cdfd8f"} Jul 21 12:28:03.958391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:03.958362 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:03.958391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:03.958380 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:03.958580 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:03.958462 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:03.958580 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:03.958476 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:28:03.958664 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:03.958576 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:28:03.958664 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:03.958641 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:28:04.129736 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:04.129645 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="b91c7ef0344ec49411d544802b7ce00963f53d7951ba8d3df9198fd43a21359f" exitCode=0 Jul 21 12:28:04.130139 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:04.129732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"b91c7ef0344ec49411d544802b7ce00963f53d7951ba8d3df9198fd43a21359f"} Jul 21 12:28:05.580574 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.580377 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:05.580574 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.580561 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:28:05.581073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.580604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:05.581073 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.580645 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret podName:0cf4f59d-684e-4266-a7f4-e9c233c4b1ff nodeName:}" failed. No retries permitted until 2026-07-21 12:28:37.58062245 +0000 UTC m=+65.103350151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret") pod "global-pull-secret-syncer-xzw56" (UID: "0cf4f59d-684e-4266-a7f4-e9c233c4b1ff") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:28:05.581073 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.580702 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:28:05.581073 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.580757 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs podName:c661bc0b-43a7-4602-9dc8-654b8095ce4a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:37.580741149 +0000 UTC m=+65.103468849 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs") pod "network-metrics-daemon-mwcl4" (UID: "c661bc0b-43a7-4602-9dc8-654b8095ce4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:28:05.681531 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.681493 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:05.681699 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.681665 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:28:05.681699 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.681686 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:28:05.681699 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.681697 2584 projected.go:196] Error preparing data for projected volume kube-api-access-npljd for pod openshift-network-diagnostics/network-check-target-dl2nt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:28:05.681803 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.681754 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd podName:e0b8c848-0bfc-4cd4-8e38-7444c6baf369 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:37.681736905 +0000 UTC m=+65.204464606 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-npljd" (UniqueName: "kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd") pod "network-check-target-dl2nt" (UID: "e0b8c848-0bfc-4cd4-8e38-7444c6baf369") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:28:05.958437 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.958406 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:05.958437 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.958405 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:05.958848 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.958542 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzw56" podUID="0cf4f59d-684e-4266-a7f4-e9c233c4b1ff" Jul 21 12:28:05.958848 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:05.958415 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:05.958848 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.958591 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dl2nt" podUID="e0b8c848-0bfc-4cd4-8e38-7444c6baf369" Jul 21 12:28:05.958848 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:05.958662 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mwcl4" podUID="c661bc0b-43a7-4602-9dc8-654b8095ce4a" Jul 21 12:28:07.282052 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.282015 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-37.ec2.internal" event="NodeReady" Jul 21 12:28:07.282891 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.282185 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 12:28:07.322648 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.321660 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:28:07.354122 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.354086 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v6svf"] Jul 21 12:28:07.354319 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.354201 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.356709 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.356679 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bv9bb\"" Jul 21 12:28:07.356858 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.356710 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 12:28:07.356858 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.356851 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 12:28:07.357031 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.356881 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 12:28:07.361672 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.361647 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 12:28:07.370076 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.370051 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w4k5k"] Jul 21 12:28:07.370202 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.370113 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.372904 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.372880 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 12:28:07.373021 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.372902 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 12:28:07.373021 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.372914 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r5psm\"" Jul 21 12:28:07.388160 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.388126 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:28:07.388324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.388171 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4k5k"] Jul 21 12:28:07.388324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.388196 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6svf"] Jul 21 12:28:07.388324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.388275 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.390794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.390737 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 12:28:07.390794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.390752 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 12:28:07.391196 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.391172 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 12:28:07.407022 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.406997 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6v6zb\"" Jul 21 12:28:07.496076 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496028 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496088 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496174 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.496302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e22981-1ca4-4f7e-be49-607fe638702a-tmp-dir\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.496302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496260 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-cert\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.496502 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496342 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496502 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496396 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2w44\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496502 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496435 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw9k8\" (UniqueName: \"kubernetes.io/projected/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-kube-api-access-dw9k8\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.496502 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496460 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm7zq\" (UniqueName: \"kubernetes.io/projected/31e22981-1ca4-4f7e-be49-607fe638702a-kube-api-access-tm7zq\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.496673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496504 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496521 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496542 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496562 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.496673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.496601 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31e22981-1ca4-4f7e-be49-607fe638702a-config-volume\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.597836 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2w44\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.597836 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597823 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw9k8\" (UniqueName: \"kubernetes.io/projected/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-kube-api-access-dw9k8\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597860 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm7zq\" (UniqueName: \"kubernetes.io/projected/31e22981-1ca4-4f7e-be49-607fe638702a-kube-api-access-tm7zq\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597891 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597948 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597971 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.597996 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31e22981-1ca4-4f7e-be49-607fe638702a-config-volume\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598062 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:07.598191 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:07.598287 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:08.098264371 +0000 UTC m=+35.620992087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598310 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e22981-1ca4-4f7e-be49-607fe638702a-tmp-dir\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598374 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-cert\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.598486 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598403 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598730 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31e22981-1ca4-4f7e-be49-607fe638702a-config-volume\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.598730 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598658 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.598819 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:07.598733 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:07.598819 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:07.598744 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:07.598819 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:07.598791 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:08.09877836 +0000 UTC m=+35.621506054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:07.598989 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.598848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e22981-1ca4-4f7e-be49-607fe638702a-tmp-dir\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.599082 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.599062 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.599428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.599402 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.603028 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.602997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.603028 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.603025 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-cert\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.603196 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.602997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.610002 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.609462 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm7zq\" (UniqueName: \"kubernetes.io/projected/31e22981-1ca4-4f7e-be49-607fe638702a-kube-api-access-tm7zq\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:07.610002 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.609699 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.610647 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.610616 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw9k8\" (UniqueName: \"kubernetes.io/projected/04fc799a-d832-4a53-9d8d-2a8bb58e0f47-kube-api-access-dw9k8\") pod \"ingress-canary-w4k5k\" (UID: \"04fc799a-d832-4a53-9d8d-2a8bb58e0f47\") " pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.611058 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.611004 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2w44\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:07.698493 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.698456 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4k5k" Jul 21 12:28:07.932276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.932245 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4k5k"] Jul 21 12:28:07.938302 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:07.938249 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fc799a_d832_4a53_9d8d_2a8bb58e0f47.slice/crio-c87a9d483a9de488d5e6aa0f5e27180c173da209554b2b6a6c3a8aa174fc4e15 WatchSource:0}: Error finding container c87a9d483a9de488d5e6aa0f5e27180c173da209554b2b6a6c3a8aa174fc4e15: Status 404 returned error can't find the container with id c87a9d483a9de488d5e6aa0f5e27180c173da209554b2b6a6c3a8aa174fc4e15 Jul 21 12:28:07.959377 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.959343 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:07.959570 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.959343 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:07.959570 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.959490 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:07.962457 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.962430 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 12:28:07.962607 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.962469 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 12:28:07.962607 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.962500 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:28:07.962831 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.962810 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mz2qd\"" Jul 21 12:28:07.963028 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.963013 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:28:07.963330 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:07.963308 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-288sp\"" Jul 21 12:28:08.104411 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:08.104379 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:08.104622 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:08.104420 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:08.104622 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:08.104555 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:08.104622 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:08.104580 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:08.104622 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:08.104586 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:08.104833 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:08.104652 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:09.104631018 +0000 UTC m=+36.627358724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:08.104833 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:08.104807 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:09.104779764 +0000 UTC m=+36.627507472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:08.140033 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:08.139945 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4k5k" event={"ID":"04fc799a-d832-4a53-9d8d-2a8bb58e0f47","Type":"ContainerStarted","Data":"c87a9d483a9de488d5e6aa0f5e27180c173da209554b2b6a6c3a8aa174fc4e15"} Jul 21 12:28:09.113288 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:09.113243 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:09.113302 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:09.113428 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:09.113446 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:09.113453 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:09.113517 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:11.113495644 +0000 UTC m=+38.636223362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:09.114244 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:09.113539 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:11.113525712 +0000 UTC m=+38.636253410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:11.133696 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:11.133671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:11.133766 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:11.133841 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:11.133854 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:11.133866 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:11.133913 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:15.13389747 +0000 UTC m=+42.656625165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:11.134037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:11.133931 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:15.133924644 +0000 UTC m=+42.656652372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:12.085899 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.085861 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74c4984944-x74sv"] Jul 21 12:28:12.096958 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.096885 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.099982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.099954 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 12:28:12.100172 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.100041 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 12:28:12.100172 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.100060 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 12:28:12.100598 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.100565 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 12:28:12.100718 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.100622 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 12:28:12.100718 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.100658 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qhks6\"" Jul 21 12:28:12.101350 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.101332 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 12:28:12.102820 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.102798 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74c4984944-x74sv"] Jul 21 12:28:12.153572 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.153539 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="b43901e5311d42c3bf0e3deb1a1c5e6b8ee17303de0d77acc166a3ff159ece91" exitCode=0 Jul 21 12:28:12.154032 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.153617 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"b43901e5311d42c3bf0e3deb1a1c5e6b8ee17303de0d77acc166a3ff159ece91"} Jul 21 12:28:12.155172 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.155148 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4k5k" event={"ID":"04fc799a-d832-4a53-9d8d-2a8bb58e0f47","Type":"ContainerStarted","Data":"cff9e082af4353686851fcd253404f5ef1a157e115e7a5054cd6e80a378dff44"} Jul 21 12:28:12.193995 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.193951 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w4k5k" podStartSLOduration=2.070725404 podStartE2EDuration="5.193931403s" totalImagesPullingTime="3.123205999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:12.193272192 +0000 UTC m=+39.715999910" watchObservedRunningTime="2026-07-21 12:28:12.193931403 +0000 UTC m=+39.716659122" Jul 21 12:28:12.243040 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.243002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-stats-auth\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.243619 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.243279 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-default-certificate\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.243619 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.243375 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.243619 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.243417 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnfrz\" (UniqueName: \"kubernetes.io/projected/0d642c82-dbe3-43f3-b083-9cb442147639-kube-api-access-hnfrz\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.243619 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.243468 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.344555 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.344524 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-default-certificate\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.344718 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.344588 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.344718 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.344693 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:12.344832 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.344722 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnfrz\" (UniqueName: \"kubernetes.io/projected/0d642c82-dbe3-43f3-b083-9cb442147639-kube-api-access-hnfrz\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.344832 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.344772 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:12.84475087 +0000 UTC m=+40.367478581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:12.344832 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.344826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.344988 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.344867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-stats-auth\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.345040 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.345009 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:12.844990356 +0000 UTC m=+40.367718069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:12.348421 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.348356 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-stats-auth\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.349542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.349519 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:12.350447 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.350430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-default-certificate\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.363957 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.363932 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnfrz\" (UniqueName: \"kubernetes.io/projected/0d642c82-dbe3-43f3-b083-9cb442147639-kube-api-access-hnfrz\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.438436 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.438399 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:12.850368 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.850330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.850541 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:12.850396 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:12.850541 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.850493 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:12.850541 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.850530 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:13.850516013 +0000 UTC m=+41.373243706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:12.850672 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:12.850556 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:13.850539643 +0000 UTC m=+41.373267337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:13.164275 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.164163 2584 generic.go:356] "Generic (PLEG): container finished" podID="cc969ba9-c0ec-4e44-8aeb-cbb820972411" containerID="5d2b63ff88a6888cc0e66fdc354a95c546b70e9e3a61c22f89e9743abf5d36ab" exitCode=0 Jul 21 12:28:13.164719 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.164304 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerDied","Data":"5d2b63ff88a6888cc0e66fdc354a95c546b70e9e3a61c22f89e9743abf5d36ab"} Jul 21 12:28:13.241595 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.241545 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:13.342453 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.342412 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:13.397535 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.397495 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:13.858069 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.858019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:13.858285 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:13.858109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:13.858285 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:13.858186 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:13.858362 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:13.858284 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:15.858264281 +0000 UTC m=+43.380991974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:13.858362 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:13.858308 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:15.858299586 +0000 UTC m=+43.381027284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:14.170583 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:14.170496 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" event={"ID":"cc969ba9-c0ec-4e44-8aeb-cbb820972411","Type":"ContainerStarted","Data":"29eeaa8cb8190de55fcecfeb42e12f7e25456abb4e43084c62ced201b9480954"} Jul 21 12:28:14.194151 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:14.194099 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l5bqq" podStartSLOduration=7.487032535 podStartE2EDuration="41.194083244s" totalImagesPullingTime="33.707050709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:14.192741923 +0000 UTC m=+41.715469639" watchObservedRunningTime="2026-07-21 12:28:14.194083244 +0000 UTC m=+41.716810960" Jul 21 12:28:14.932901 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:14.932865 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:14.985269 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:14.985230 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:15.168029 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.167987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:15.168029 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.168034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:15.168326 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.168147 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:15.168326 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.168157 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:15.168326 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.168171 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:15.168326 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.168258 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:23.16823855 +0000 UTC m=+50.690966266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:15.168326 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.168287 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:23.16827405 +0000 UTC m=+50.691001757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:15.463126 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.463086 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:15.528476 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.528438 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:15.874088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.873988 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:15.874088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:15.874065 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:15.874356 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.874154 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:15.874356 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.874206 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:19.874186979 +0000 UTC m=+47.396914686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:15.874356 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:15.874261 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:19.874250701 +0000 UTC m=+47.396978399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:16.860979 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:16.860939 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994"] Jul 21 12:28:16.884405 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:16.884373 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994"] Jul 21 12:28:16.884555 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:16.884474 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" Jul 21 12:28:16.887500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:16.887477 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dvtpf\"" Jul 21 12:28:16.982549 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:16.982503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l44d\" (UniqueName: \"kubernetes.io/projected/edb83309-af9c-478f-9471-7ec6100590c4-kube-api-access-2l44d\") pod \"network-check-source-7bc5b6bf85-l9994\" (UID: \"edb83309-af9c-478f-9471-7ec6100590c4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" Jul 21 12:28:17.083003 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:17.082965 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l44d\" (UniqueName: \"kubernetes.io/projected/edb83309-af9c-478f-9471-7ec6100590c4-kube-api-access-2l44d\") pod \"network-check-source-7bc5b6bf85-l9994\" (UID: \"edb83309-af9c-478f-9471-7ec6100590c4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" Jul 21 12:28:17.097095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:17.097065 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l44d\" (UniqueName: \"kubernetes.io/projected/edb83309-af9c-478f-9471-7ec6100590c4-kube-api-access-2l44d\") pod \"network-check-source-7bc5b6bf85-l9994\" (UID: \"edb83309-af9c-478f-9471-7ec6100590c4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" Jul 21 12:28:17.193118 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:17.193085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" Jul 21 12:28:17.347096 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:17.347064 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994"] Jul 21 12:28:17.350465 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:17.350431 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb83309_af9c_478f_9471_7ec6100590c4.slice/crio-f2ddbe205d045bf61f595fba2a048c6a4523a35c1c1cde28a5a38f8582ab6c75 WatchSource:0}: Error finding container f2ddbe205d045bf61f595fba2a048c6a4523a35c1c1cde28a5a38f8582ab6c75: Status 404 returned error can't find the container with id f2ddbe205d045bf61f595fba2a048c6a4523a35c1c1cde28a5a38f8582ab6c75 Jul 21 12:28:18.181981 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:18.181936 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" event={"ID":"edb83309-af9c-478f-9471-7ec6100590c4","Type":"ContainerStarted","Data":"f2ddbe205d045bf61f595fba2a048c6a4523a35c1c1cde28a5a38f8582ab6c75"} Jul 21 12:28:18.277820 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:18.277774 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:18.332843 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:18.332704 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.099653 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.099592 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.187224 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.187177 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.275440 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.275401 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-27cp7"] Jul 21 12:28:19.302419 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.302386 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27cp7"] Jul 21 12:28:19.302599 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.302511 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.306959 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.306890 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 12:28:19.307297 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.307273 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 12:28:19.307429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.307410 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c9cc7\"" Jul 21 12:28:19.307932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.307913 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 12:28:19.307932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.307927 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 12:28:19.336451 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.336409 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.408088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.407982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-crio-socket\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.408088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.408041 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.408088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.408075 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-data-volume\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.408353 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.408172 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.408353 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.408254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ndc8\" (UniqueName: \"kubernetes.io/projected/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-api-access-4ndc8\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.438229 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.438185 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.509322 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509285 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-crio-socket\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509340 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-data-volume\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509407 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ndc8\" (UniqueName: \"kubernetes.io/projected/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-api-access-4ndc8\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509515 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-crio-socket\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.509531 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:28:19.509547 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.509531 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:28:19.509819 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.509616 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:20.009595472 +0000 UTC m=+47.532323169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:28:19.509819 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.509635 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:20.009627429 +0000 UTC m=+47.532355123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : configmap "kube-rbac-proxy" not found Jul 21 12:28:19.509819 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.509726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-data-volume\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.522107 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.522074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ndc8\" (UniqueName: \"kubernetes.io/projected/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-api-access-4ndc8\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:19.528909 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.528879 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.589032 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.588965 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.808900 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.808863 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.911289 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.911245 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:19.913149 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.913126 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:19.913309 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:19.913200 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:19.913373 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.913304 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:19.913373 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.913358 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:27.913342164 +0000 UTC m=+55.436069860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:19.913484 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:19.913377 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:27.913370863 +0000 UTC m=+55.436098557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:20.014143 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:20.014096 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:20.014346 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:20.014170 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:20.014346 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:20.014273 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:28:20.014346 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:20.014281 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:28:20.014491 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:20.014353 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:21.014330393 +0000 UTC m=+48.537058096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:28:20.014491 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:20.014387 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:21.014376627 +0000 UTC m=+48.537104320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : configmap "kube-rbac-proxy" not found Jul 21 12:28:21.024371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:21.024332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:21.024763 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:21.024388 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:21.024763 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:21.024496 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:28:21.024763 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:21.024516 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:28:21.024763 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:21.024571 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:23.024555018 +0000 UTC m=+50.547282713 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : configmap "kube-rbac-proxy" not found Jul 21 12:28:21.024763 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:21.024587 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:23.024580475 +0000 UTC m=+50.547308168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:28:21.431464 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:21.431425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:21.446373 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:21.446347 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r27b5_f8b5a32b-9781-4981-bb66-3863b4ade135/dns-node-resolver/0.log" Jul 21 12:28:22.047463 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:22.047429 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7rvj2_e335be90-a7bf-4077-a72a-694dbddcb799/node-ca/0.log" Jul 21 12:28:22.194578 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:22.194531 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" event={"ID":"edb83309-af9c-478f-9471-7ec6100590c4","Type":"ContainerStarted","Data":"f8e04c04730205428b31dc0a898bb4c26991242aeadff4881f80655cc4293e25"} Jul 21 12:28:22.216254 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:22.216173 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-l9994" podStartSLOduration=2.442566028 podStartE2EDuration="6.216151547s" totalImagesPullingTime="3.773585519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:22.214842429 +0000 UTC m=+49.737570148" watchObservedRunningTime="2026-07-21 12:28:22.216151547 +0000 UTC m=+49.738879266" Jul 21 12:28:22.299871 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:22.299780 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:22.668805 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:22.668702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:23.039935 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.039883 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:23.039935 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.039941 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:23.040167 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.040040 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:28:23.040167 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.040124 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:27.040100659 +0000 UTC m=+54.562828353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:28:23.040671 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.040651 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:23.106821 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.106784 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:23.241508 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.241463 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.241517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.241576 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.241601 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fbb7d9bcb-988vt: secret "image-registry-tls" not found Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.241657 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.241662 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls podName:570997d6-e8ac-40c8-a663-f39a6fa394ed nodeName:}" failed. No retries permitted until 2026-07-21 12:28:39.241644747 +0000 UTC m=+66.764372440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls") pod "image-registry-6fbb7d9bcb-988vt" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed") : secret "image-registry-tls" not found Jul 21 12:28:23.241722 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:23.241719 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls podName:31e22981-1ca4-4f7e-be49-607fe638702a nodeName:}" failed. No retries permitted until 2026-07-21 12:28:39.241702847 +0000 UTC m=+66.764430545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls") pod "dns-default-v6svf" (UID: "31e22981-1ca4-4f7e-be49-607fe638702a") : secret "dns-default-metrics-tls" not found Jul 21 12:28:23.247013 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:23.246990 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4k5k_04fc799a-d832-4a53-9d8d-2a8bb58e0f47/serve-healthcheck-canary/0.log" Jul 21 12:28:26.050980 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.050949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.176546 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.176513 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.368815 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.368722 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.470451 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.470416 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.569237 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.569188 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.669808 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.669713 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:26.922136 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:26.922048 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:27.072507 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:27.072457 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:27.072952 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:27.072620 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:28:27.072952 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:27.072707 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls podName:6ffcfe75-064b-43de-a0ed-00b0414e0f3f nodeName:}" failed. No retries permitted until 2026-07-21 12:28:35.072687322 +0000 UTC m=+62.595415088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27cp7" (UID: "6ffcfe75-064b-43de-a0ed-00b0414e0f3f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:28:27.980927 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:27.980875 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:27.981130 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:27.980953 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:27.981130 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:27.981041 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:28:27.981130 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:27.981087 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:43.981072509 +0000 UTC m=+71.503800204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : configmap references non-existent config key: service-ca.crt Jul 21 12:28:27.981130 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:27.981102 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs podName:0d642c82-dbe3-43f3-b083-9cb442147639 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:43.98109541 +0000 UTC m=+71.503823104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs") pod "router-default-74c4984944-x74sv" (UID: "0d642c82-dbe3-43f3-b083-9cb442147639") : secret "router-metrics-certs-default" not found Jul 21 12:28:28.070982 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:28.070941 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:28.919396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:28.919364 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:28.979289 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:28.979252 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:29.866844 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:29.866807 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:29.919975 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:29.919940 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:30.292439 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:30.292406 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:30.465014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:30.464974 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:30.565405 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:30.565316 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:30.665605 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:30.665568 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:30.723536 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:30.723497 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:31.063311 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:31.063271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:31.873083 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:31.873048 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:32.066064 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:32.066029 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:32.470146 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:32.470107 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:32.570309 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:32.570270 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:32.669796 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:32.669760 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:33.060341 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:33.060304 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:33.465022 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:33.464988 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:33.968678 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:33.968630 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:34.140457 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:34.140425 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlm2m" Jul 21 12:28:34.584509 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:34.584470 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:34.685781 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:34.685745 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:34.896095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:34.896004 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:35.141317 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.141273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:35.143591 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.143559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ffcfe75-064b-43de-a0ed-00b0414e0f3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27cp7\" (UID: \"6ffcfe75-064b-43de-a0ed-00b0414e0f3f\") " pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:35.215538 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.215511 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c9cc7\"" Jul 21 12:28:35.223382 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.223357 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27cp7" Jul 21 12:28:35.261051 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.261014 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:35.354132 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.354094 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27cp7"] Jul 21 12:28:35.358583 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:35.358552 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffcfe75_064b_43de_a0ed_00b0414e0f3f.slice/crio-5a861b9feb471a1b35c3ac93b814a54deff3ce3b43080f7a11b814dac0229408 WatchSource:0}: Error finding container 5a861b9feb471a1b35c3ac93b814a54deff3ce3b43080f7a11b814dac0229408: Status 404 returned error can't find the container with id 5a861b9feb471a1b35c3ac93b814a54deff3ce3b43080f7a11b814dac0229408 Jul 21 12:28:35.679225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.679161 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:35.873811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:35.873760 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:36.228289 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:36.228249 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27cp7" event={"ID":"6ffcfe75-064b-43de-a0ed-00b0414e0f3f","Type":"ContainerStarted","Data":"5a861b9feb471a1b35c3ac93b814a54deff3ce3b43080f7a11b814dac0229408"} Jul 21 12:28:36.330578 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:36.330537 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:36.479625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:36.479535 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:36.869854 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:36.869751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:37.663270 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.663228 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:37.663485 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.663319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:37.666057 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.666023 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:28:37.666057 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.666023 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:28:37.677077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.677051 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c661bc0b-43a7-4602-9dc8-654b8095ce4a-metrics-certs\") pod \"network-metrics-daemon-mwcl4\" (UID: \"c661bc0b-43a7-4602-9dc8-654b8095ce4a\") " pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:37.677252 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.677147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cf4f59d-684e-4266-a7f4-e9c233c4b1ff-original-pull-secret\") pod \"global-pull-secret-syncer-xzw56\" (UID: \"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff\") " pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:37.686336 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.686301 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzw56" Jul 21 12:28:37.764538 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.764485 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:37.767593 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.767561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npljd\" (UniqueName: \"kubernetes.io/projected/e0b8c848-0bfc-4cd4-8e38-7444c6baf369-kube-api-access-npljd\") pod \"network-check-target-dl2nt\" (UID: \"e0b8c848-0bfc-4cd4-8e38-7444c6baf369\") " pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:37.975384 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.975349 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-288sp\"" Jul 21 12:28:37.981895 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.981863 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mz2qd\"" Jul 21 12:28:37.983559 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.983536 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mwcl4" Jul 21 12:28:37.989555 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:37.989521 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:39.277869 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.277827 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:39.278327 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.277888 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:39.280623 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.280584 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e22981-1ca4-4f7e-be49-607fe638702a-metrics-tls\") pod \"dns-default-v6svf\" (UID: \"31e22981-1ca4-4f7e-be49-607fe638702a\") " pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:39.280747 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.280643 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"image-registry-6fbb7d9bcb-988vt\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:39.468064 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.468030 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bv9bb\"" Jul 21 12:28:39.476099 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.476071 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:39.483227 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.483179 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r5psm\"" Jul 21 12:28:39.490790 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:39.490758 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:40.246830 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.246772 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27cp7" event={"ID":"6ffcfe75-064b-43de-a0ed-00b0414e0f3f","Type":"ContainerStarted","Data":"a63ffdeb847e7db6ab1144bb3641418949f15f51a892ee6b51711f42d6a07345"} Jul 21 12:28:40.286424 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.286383 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mwcl4"] Jul 21 12:28:40.292299 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:40.292258 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc661bc0b_43a7_4602_9dc8_654b8095ce4a.slice/crio-7af01896c7b7287850f8722e4857e040b1756cd39457dc97b136b2c9113c68de WatchSource:0}: Error finding container 7af01896c7b7287850f8722e4857e040b1756cd39457dc97b136b2c9113c68de: Status 404 returned error can't find the container with id 7af01896c7b7287850f8722e4857e040b1756cd39457dc97b136b2c9113c68de Jul 21 12:28:40.503596 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.503483 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzw56"] Jul 21 12:28:40.504242 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.504201 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dl2nt"] Jul 21 12:28:40.507114 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.507094 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6svf"] Jul 21 12:28:40.507343 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:40.507288 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:28:40.507343 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:40.507302 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b8c848_0bfc_4cd4_8e38_7444c6baf369.slice/crio-866723a2bafef97f4c11517974639f49ba42c0b24bd2868a675b9badf7b91846 WatchSource:0}: Error finding container 866723a2bafef97f4c11517974639f49ba42c0b24bd2868a675b9badf7b91846: Status 404 returned error can't find the container with id 866723a2bafef97f4c11517974639f49ba42c0b24bd2868a675b9badf7b91846 Jul 21 12:28:40.507629 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:40.507605 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf4f59d_684e_4266_a7f4_e9c233c4b1ff.slice/crio-e341cc9f5b5c1afbf3c59d05195b22db4d4742e309f034c235f64d3b54bac038 WatchSource:0}: Error finding container e341cc9f5b5c1afbf3c59d05195b22db4d4742e309f034c235f64d3b54bac038: Status 404 returned error can't find the container with id e341cc9f5b5c1afbf3c59d05195b22db4d4742e309f034c235f64d3b54bac038 Jul 21 12:28:40.509507 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:40.509483 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570997d6_e8ac_40c8_a663_f39a6fa394ed.slice/crio-f820c36dda7e471f63ba17aa0e642340d431a327760a0b5450747bcb2d8a6ef9 WatchSource:0}: Error finding container f820c36dda7e471f63ba17aa0e642340d431a327760a0b5450747bcb2d8a6ef9: Status 404 returned error can't find the container with id f820c36dda7e471f63ba17aa0e642340d431a327760a0b5450747bcb2d8a6ef9 Jul 21 12:28:40.510940 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:40.510917 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e22981_1ca4_4f7e_be49_607fe638702a.slice/crio-1fe480083400ed8dbbae2d6376a6c8aa9ed18326bf102f3a2839307f74d87bad WatchSource:0}: Error finding container 1fe480083400ed8dbbae2d6376a6c8aa9ed18326bf102f3a2839307f74d87bad: Status 404 returned error can't find the container with id 1fe480083400ed8dbbae2d6376a6c8aa9ed18326bf102f3a2839307f74d87bad Jul 21 12:28:41.253234 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.253127 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6svf" event={"ID":"31e22981-1ca4-4f7e-be49-607fe638702a","Type":"ContainerStarted","Data":"1fe480083400ed8dbbae2d6376a6c8aa9ed18326bf102f3a2839307f74d87bad"} Jul 21 12:28:41.256898 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.256805 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dl2nt" event={"ID":"e0b8c848-0bfc-4cd4-8e38-7444c6baf369","Type":"ContainerStarted","Data":"a482be4dd7c68ecbc502d70fb45d1b392cefe1f74b9fbde9f633820fec2f4ceb"} Jul 21 12:28:41.256898 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.256864 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dl2nt" event={"ID":"e0b8c848-0bfc-4cd4-8e38-7444c6baf369","Type":"ContainerStarted","Data":"866723a2bafef97f4c11517974639f49ba42c0b24bd2868a675b9badf7b91846"} Jul 21 12:28:41.258111 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.257657 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:28:41.259388 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.259296 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mwcl4" event={"ID":"c661bc0b-43a7-4602-9dc8-654b8095ce4a","Type":"ContainerStarted","Data":"7af01896c7b7287850f8722e4857e040b1756cd39457dc97b136b2c9113c68de"} Jul 21 12:28:41.262235 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.261888 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" event={"ID":"570997d6-e8ac-40c8-a663-f39a6fa394ed","Type":"ContainerStarted","Data":"b1743db1794ccd28d0d208f2de068ddb051006a1fb7d61ad13cbc822a78ce7bb"} Jul 21 12:28:41.262235 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.261934 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" event={"ID":"570997d6-e8ac-40c8-a663-f39a6fa394ed","Type":"ContainerStarted","Data":"f820c36dda7e471f63ba17aa0e642340d431a327760a0b5450747bcb2d8a6ef9"} Jul 21 12:28:41.262235 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.262136 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:28:41.263699 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.263628 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzw56" event={"ID":"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff","Type":"ContainerStarted","Data":"e341cc9f5b5c1afbf3c59d05195b22db4d4742e309f034c235f64d3b54bac038"} Jul 21 12:28:41.275744 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.275627 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dl2nt" podStartSLOduration=69.275603119 podStartE2EDuration="1m9.275603119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:41.275516988 +0000 UTC m=+68.798244706" watchObservedRunningTime="2026-07-21 12:28:41.275603119 +0000 UTC m=+68.798330815" Jul 21 12:28:41.295988 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.295719 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" podStartSLOduration=68.295693871 podStartE2EDuration="1m8.295693871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:41.294378579 +0000 UTC m=+68.817106299" watchObservedRunningTime="2026-07-21 12:28:41.295693871 +0000 UTC m=+68.818421588" Jul 21 12:28:41.650341 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.650181 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:41.755428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.755391 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:41.850894 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.850811 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:41.950398 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:41.950355 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:42.105005 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.104950 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:42.170623 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.170593 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:42.271597 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.271487 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mwcl4" event={"ID":"c661bc0b-43a7-4602-9dc8-654b8095ce4a","Type":"ContainerStarted","Data":"31d197fc55d530a63cfd29a87b6925c497836e392aebda3bb21be1c96b40247b"} Jul 21 12:28:42.271597 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.271565 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mwcl4" event={"ID":"c661bc0b-43a7-4602-9dc8-654b8095ce4a","Type":"ContainerStarted","Data":"4cde8e1d5a64a5840819a2e2c8898e885d2c21fbc3b8cb0cf0d43882f5a94a51"} Jul 21 12:28:42.289323 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.289259 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mwcl4" podStartSLOduration=68.928029553 podStartE2EDuration="1m10.289234906s" totalImagesPullingTime="1.361205353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:42.287928731 +0000 UTC m=+69.810656446" watchObservedRunningTime="2026-07-21 12:28:42.289234906 +0000 UTC m=+69.811962617" Jul 21 12:28:42.299177 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.299146 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:42.369374 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.369328 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:42.706671 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:42.706628 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:43.305745 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:43.305527 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:43.508622 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:43.508580 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:43.572099 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:43.571973 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:44.021371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.021332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:44.021577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.021426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:44.022242 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.022195 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d642c82-dbe3-43f3-b083-9cb442147639-service-ca-bundle\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:44.024194 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.024165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d642c82-dbe3-43f3-b083-9cb442147639-metrics-certs\") pod \"router-default-74c4984944-x74sv\" (UID: \"0d642c82-dbe3-43f3-b083-9cb442147639\") " pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:44.211741 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.211701 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qhks6\"" Jul 21 12:28:44.219401 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.219367 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:44.288751 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.288661 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-4xmlr"] Jul 21 12:28:44.305772 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.305740 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:28:44.310415 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.310380 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-x7bwc\"" Jul 21 12:28:44.310865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.310843 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 12:28:44.315457 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.315431 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-4xmlr"] Jul 21 12:28:44.316607 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.316088 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 12:28:44.371583 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.371531 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:44.425543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.425503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrngq\" (UniqueName: \"kubernetes.io/projected/0f13f8cd-03f0-4664-8198-220f6bfb538b-kube-api-access-vrngq\") pod \"downloads-5bc67dcd46-4xmlr\" (UID: \"0f13f8cd-03f0-4664-8198-220f6bfb538b\") " pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:28:44.461428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.461394 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:44.527015 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.526951 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrngq\" (UniqueName: \"kubernetes.io/projected/0f13f8cd-03f0-4664-8198-220f6bfb538b-kube-api-access-vrngq\") pod \"downloads-5bc67dcd46-4xmlr\" (UID: \"0f13f8cd-03f0-4664-8198-220f6bfb538b\") " pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:28:44.536872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.536838 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrngq\" (UniqueName: \"kubernetes.io/projected/0f13f8cd-03f0-4664-8198-220f6bfb538b-kube-api-access-vrngq\") pod \"downloads-5bc67dcd46-4xmlr\" (UID: \"0f13f8cd-03f0-4664-8198-220f6bfb538b\") " pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:28:44.562518 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.562428 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:44.620487 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.620435 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:28:44.663304 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:44.663263 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:45.467187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:45.467137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:45.536701 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:45.536664 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:46.633146 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:46.633027 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-4xmlr"] Jul 21 12:28:46.639522 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:46.639489 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f13f8cd_03f0_4664_8198_220f6bfb538b.slice/crio-65d49633003ac820925c39cf6cc8336466a223c7dea43a72cb191ea7e13c09bc WatchSource:0}: Error finding container 65d49633003ac820925c39cf6cc8336466a223c7dea43a72cb191ea7e13c09bc: Status 404 returned error can't find the container with id 65d49633003ac820925c39cf6cc8336466a223c7dea43a72cb191ea7e13c09bc Jul 21 12:28:46.657381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:46.657299 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74c4984944-x74sv"] Jul 21 12:28:46.661016 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:46.660986 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d642c82_dbe3_43f3_b083_9cb442147639.slice/crio-ae4014aee1596775f15f3a3090924bd150727160dbf3fb17552db3e167362877 WatchSource:0}: Error finding container ae4014aee1596775f15f3a3090924bd150727160dbf3fb17552db3e167362877: Status 404 returned error can't find the container with id ae4014aee1596775f15f3a3090924bd150727160dbf3fb17552db3e167362877 Jul 21 12:28:47.294438 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.294394 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-4xmlr" event={"ID":"0f13f8cd-03f0-4664-8198-220f6bfb538b","Type":"ContainerStarted","Data":"65d49633003ac820925c39cf6cc8336466a223c7dea43a72cb191ea7e13c09bc"} Jul 21 12:28:47.296803 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.296503 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzw56" event={"ID":"0cf4f59d-684e-4266-a7f4-e9c233c4b1ff","Type":"ContainerStarted","Data":"7af430ca99ba148b74e4a2c9b4c93452966913ca278da3d9c806aacc942cdd8c"} Jul 21 12:28:47.298934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.298898 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6svf" event={"ID":"31e22981-1ca4-4f7e-be49-607fe638702a","Type":"ContainerStarted","Data":"bce40ad8d096870c563c3b0b2ce8ce5cc455bb56d380a83c6538933a525b5eda"} Jul 21 12:28:47.299095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.298960 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6svf" event={"ID":"31e22981-1ca4-4f7e-be49-607fe638702a","Type":"ContainerStarted","Data":"f70934d4fc6d61ca0994779f1d704af8b71e727fd88d232a3c6be4aa5adf0d81"} Jul 21 12:28:47.299095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.298999 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:47.301185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.301096 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74c4984944-x74sv" event={"ID":"0d642c82-dbe3-43f3-b083-9cb442147639","Type":"ContainerStarted","Data":"97c591edc1844c943933f1819532cb68feba6febe48522bca4319fa596b99874"} Jul 21 12:28:47.301185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.301154 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74c4984944-x74sv" event={"ID":"0d642c82-dbe3-43f3-b083-9cb442147639","Type":"ContainerStarted","Data":"ae4014aee1596775f15f3a3090924bd150727160dbf3fb17552db3e167362877"} Jul 21 12:28:47.303519 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.303482 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27cp7" event={"ID":"6ffcfe75-064b-43de-a0ed-00b0414e0f3f","Type":"ContainerStarted","Data":"54ff5634fa0025d7fd771d1ec8a137f19e143123b5ba1f9a9046db2d9178fc4a"} Jul 21 12:28:47.312027 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.311971 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xzw56" podStartSLOduration=68.39133629 podStartE2EDuration="1m14.311952212s" totalImagesPullingTime="5.920615922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:47.31106446 +0000 UTC m=+74.833792178" watchObservedRunningTime="2026-07-21 12:28:47.311952212 +0000 UTC m=+74.834679976" Jul 21 12:28:47.329407 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.329336 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74c4984944-x74sv" podStartSLOduration=35.329312168 podStartE2EDuration="35.329312168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:47.327891478 +0000 UTC m=+74.850619195" watchObservedRunningTime="2026-07-21 12:28:47.329312168 +0000 UTC m=+74.852039885" Jul 21 12:28:47.345252 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.344960 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v6svf" podStartSLOduration=34.425302505 podStartE2EDuration="40.344933771s" totalImagesPullingTime="5.919631266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:47.343480858 +0000 UTC m=+74.866208575" watchObservedRunningTime="2026-07-21 12:28:47.344933771 +0000 UTC m=+74.867661492" Jul 21 12:28:47.472566 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.472526 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:47.636112 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:47.635997 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:48.220326 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:48.220286 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:48.223888 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:48.223859 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:48.312144 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:48.312103 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:48.312923 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:48.312895 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74c4984944-x74sv" Jul 21 12:28:49.537910 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.537868 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:49.552710 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.552676 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:28:49.564428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.564386 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.569603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.569563 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 12:28:49.570807 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.570740 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:28:49.572133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.571263 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dk8lv\"" Jul 21 12:28:49.572133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.571335 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 12:28:49.572133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.571558 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 12:28:49.572133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.571777 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 12:28:49.572133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.571972 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 12:28:49.615907 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.615869 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:49.669258 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.669186 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:49.673485 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673454 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.673660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673496 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.673660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673517 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.673779 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673652 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.673779 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673707 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.673885 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.673793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jr6\" (UniqueName: \"kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775025 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.774976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775204 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775054 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775204 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jr6\" (UniqueName: \"kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775204 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775150 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775204 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.775442 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775238 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.776004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.775973 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.776260 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.776193 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.776394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.776351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.778109 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.778082 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.778814 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.778789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.789861 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.789758 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jr6\" (UniqueName: \"kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6\") pod \"console-7d64c9b84-whd8g\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:49.878155 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:49.878117 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:50.542346 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:50.542307 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:28:50.547249 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:50.547194 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21d9076_e82f_437b_98f6_8aa15fc3d92b.slice/crio-9a515c0ce170e77e051e49cf030ca34f03ceebc68c1c2c9157e89e5afb69f28c WatchSource:0}: Error finding container 9a515c0ce170e77e051e49cf030ca34f03ceebc68c1c2c9157e89e5afb69f28c: Status 404 returned error can't find the container with id 9a515c0ce170e77e051e49cf030ca34f03ceebc68c1c2c9157e89e5afb69f28c Jul 21 12:28:51.273876 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:51.273840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:51.325642 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:51.325600 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27cp7" event={"ID":"6ffcfe75-064b-43de-a0ed-00b0414e0f3f","Type":"ContainerStarted","Data":"b99a8f8fab300a5aee38b9bf93da731e62e761bb24dca2011705fe5e190adaa2"} Jul 21 12:28:51.327727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:51.327691 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64c9b84-whd8g" event={"ID":"d21d9076-e82f-437b-98f6-8aa15fc3d92b","Type":"ContainerStarted","Data":"9a515c0ce170e77e051e49cf030ca34f03ceebc68c1c2c9157e89e5afb69f28c"} Jul 21 12:28:51.345527 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:51.345454 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-27cp7" podStartSLOduration=17.465687716 podStartE2EDuration="32.345428629s" totalImagesPullingTime="14.879740913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:28:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:51.344269789 +0000 UTC m=+78.866997506" watchObservedRunningTime="2026-07-21 12:28:51.345428629 +0000 UTC m=+78.868156346" Jul 21 12:28:51.349479 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:51.349446 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:53.293839 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:53.293777 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:53.394922 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:53.394887 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:53.456823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:53.456777 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:54.343796 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:54.343753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64c9b84-whd8g" event={"ID":"d21d9076-e82f-437b-98f6-8aa15fc3d92b","Type":"ContainerStarted","Data":"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445"} Jul 21 12:28:54.362029 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:54.361950 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d64c9b84-whd8g" podStartSLOduration=2.356093901 podStartE2EDuration="5.361927142s" totalImagesPullingTime="3.005833241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:54.361185757 +0000 UTC m=+81.883913475" watchObservedRunningTime="2026-07-21 12:28:54.361927142 +0000 UTC m=+81.884654861" Jul 21 12:28:55.178160 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.178117 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:55.280795 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.280745 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:55.316950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.316917 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz"] Jul 21 12:28:55.321433 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.321399 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:55.323713 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.323683 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4prk4\"" Jul 21 12:28:55.323850 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.323782 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 12:28:55.328104 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.328063 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz"] Jul 21 12:28:55.377985 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.377952 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:55.418478 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.418411 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9xftz\" (UID: \"2ed13308-0587-4739-b04c-042123d315d7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:55.519474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.519376 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9xftz\" (UID: \"2ed13308-0587-4739-b04c-042123d315d7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:55.519634 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:55.519550 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 21 12:28:55.519688 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:28:55.519647 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates podName:2ed13308-0587-4739-b04c-042123d315d7 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:56.019621947 +0000 UTC m=+83.542349656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-9xftz" (UID: "2ed13308-0587-4739-b04c-042123d315d7") : secret "prometheus-operator-admission-webhook-tls" not found Jul 21 12:28:55.595585 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:55.595513 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:56.024533 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:56.024490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9xftz\" (UID: \"2ed13308-0587-4739-b04c-042123d315d7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:56.027494 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:56.027465 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ed13308-0587-4739-b04c-042123d315d7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9xftz\" (UID: \"2ed13308-0587-4739-b04c-042123d315d7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:56.233887 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:56.233845 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:28:56.379094 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:56.379057 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz"] Jul 21 12:28:56.382922 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:28:56.382893 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed13308_0587_4739_b04c_042123d315d7.slice/crio-15b004184a5f4e0c73ad3e16a2c5c56814a6ba0b4b9c6975c1234e83592a393b WatchSource:0}: Error finding container 15b004184a5f4e0c73ad3e16a2c5c56814a6ba0b4b9c6975c1234e83592a393b: Status 404 returned error can't find the container with id 15b004184a5f4e0c73ad3e16a2c5c56814a6ba0b4b9c6975c1234e83592a393b Jul 21 12:28:56.741025 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:56.740992 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:57.252655 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.252617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:57.314110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.314075 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v6svf" Jul 21 12:28:57.319735 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.319706 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:28:57.322812 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.322788 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.331109 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.331072 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 12:28:57.338280 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.338253 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:28:57.358335 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.358297 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" event={"ID":"2ed13308-0587-4739-b04c-042123d315d7","Type":"ContainerStarted","Data":"15b004184a5f4e0c73ad3e16a2c5c56814a6ba0b4b9c6975c1234e83592a393b"} Jul 21 12:28:57.435608 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.435608 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435614 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.436169 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435648 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.436169 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.436169 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435772 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.436169 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435802 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.436169 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.435879 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl5cr\" (UniqueName: \"kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536565 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536469 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl5cr\" (UniqueName: \"kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536565 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536617 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536637 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536672 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.536798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.536697 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.537584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.537558 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.537733 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.537567 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.537822 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.537727 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.537822 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.537742 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.539842 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.539810 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.539971 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.539910 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.549379 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.549349 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl5cr\" (UniqueName: \"kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr\") pod \"console-5fb67d6d95-8kw22\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:57.635116 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:57.634852 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:28:59.339535 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:59.339493 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:59.443754 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:59.443707 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:28:59.878869 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:59.878828 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:59.879085 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:59.878911 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:28:59.884533 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:28:59.884507 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:29:00.374102 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:00.374071 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:29:00.462010 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:00.461871 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:00.565419 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:00.565376 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:02.277439 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:02.277411 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:29:02.447513 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:02.447463 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:02.582835 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:02.582739 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:02.936596 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:02.936546 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:03.099654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:03.099615 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:03.941043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:03.941003 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:04.045043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:04.045004 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:05.061849 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.061811 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:29:05.065440 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:05.065403 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf726c98e_8e26_4392_bf25_a620b1cd7d79.slice/crio-2c84445b2548e56c9b40fece7b8a6dccd6fab53abd475a11bf0e38bcb774ec71 WatchSource:0}: Error finding container 2c84445b2548e56c9b40fece7b8a6dccd6fab53abd475a11bf0e38bcb774ec71: Status 404 returned error can't find the container with id 2c84445b2548e56c9b40fece7b8a6dccd6fab53abd475a11bf0e38bcb774ec71 Jul 21 12:29:05.393057 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.392956 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" event={"ID":"2ed13308-0587-4739-b04c-042123d315d7","Type":"ContainerStarted","Data":"aee7e27ae2607f40a2b1a20700e4ad6ef661b6685b49ccabd91c036e4e057f23"} Jul 21 12:29:05.393273 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.393135 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:29:05.394953 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.394920 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-4xmlr" event={"ID":"0f13f8cd-03f0-4664-8198-220f6bfb538b","Type":"ContainerStarted","Data":"483de2b030793dde9483547317b8ad620b3450c069b7259b49400405ee6afec0"} Jul 21 12:29:05.395347 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.395272 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:29:05.397231 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.397182 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb67d6d95-8kw22" event={"ID":"f726c98e-8e26-4392-bf25-a620b1cd7d79","Type":"ContainerStarted","Data":"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c"} Jul 21 12:29:05.397336 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.397256 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb67d6d95-8kw22" event={"ID":"f726c98e-8e26-4392-bf25-a620b1cd7d79","Type":"ContainerStarted","Data":"2c84445b2548e56c9b40fece7b8a6dccd6fab53abd475a11bf0e38bcb774ec71"} Jul 21 12:29:05.399321 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.399293 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" Jul 21 12:29:05.401912 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.401885 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-4xmlr" Jul 21 12:29:05.408887 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.408830 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9xftz" podStartSLOduration=1.863881315 podStartE2EDuration="10.408813796s" totalImagesPullingTime="8.544932481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:05.407667579 +0000 UTC m=+92.930395298" watchObservedRunningTime="2026-07-21 12:29:05.408813796 +0000 UTC m=+92.931541512" Jul 21 12:29:05.425763 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.425699 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fb67d6d95-8kw22" podStartSLOduration=8.425677439 podStartE2EDuration="8.425677439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:05.424923471 +0000 UTC m=+92.947651200" watchObservedRunningTime="2026-07-21 12:29:05.425677439 +0000 UTC m=+92.948405232" Jul 21 12:29:05.442432 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:05.442363 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-4xmlr" podStartSLOduration=3.133630208 podStartE2EDuration="21.442339658s" totalImagesPullingTime="18.30870945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:05.440558646 +0000 UTC m=+92.963286390" watchObservedRunningTime="2026-07-21 12:29:05.442339658 +0000 UTC m=+92.965067374" Jul 21 12:29:06.345670 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.345630 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:06.394409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.394369 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf"] Jul 21 12:29:06.425693 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.425634 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf"] Jul 21 12:29:06.425879 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.425766 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.428474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.428442 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 12:29:06.428474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.428461 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 12:29:06.428474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.428442 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 12:29:06.429464 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.429443 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 12:29:06.429623 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.429517 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 12:29:06.429723 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.429525 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gxzc9\"" Jul 21 12:29:06.447406 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.447364 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:06.547357 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.547314 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:06.621147 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.621043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85cj\" (UniqueName: \"kubernetes.io/projected/e73ddc9e-fc8a-4612-87b4-e869b304199b-kube-api-access-m85cj\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.621147 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.621129 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.621410 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.621155 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ddc9e-fc8a-4612-87b4-e869b304199b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.621410 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.621202 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.650266 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.650195 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:06.722302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.722261 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85cj\" (UniqueName: \"kubernetes.io/projected/e73ddc9e-fc8a-4612-87b4-e869b304199b-kube-api-access-m85cj\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.722500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.722330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.722500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.722360 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ddc9e-fc8a-4612-87b4-e869b304199b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.722500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.722406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.723194 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.723165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ddc9e-fc8a-4612-87b4-e869b304199b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.725425 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.725397 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.725655 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.725633 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e73ddc9e-fc8a-4612-87b4-e869b304199b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.732290 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.732263 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85cj\" (UniqueName: \"kubernetes.io/projected/e73ddc9e-fc8a-4612-87b4-e869b304199b-kube-api-access-m85cj\") pod \"prometheus-operator-5945b5cfd-8p6jf\" (UID: \"e73ddc9e-fc8a-4612-87b4-e869b304199b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.737955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.737919 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" Jul 21 12:29:06.916517 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:06.916225 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf"] Jul 21 12:29:06.920057 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:06.920019 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73ddc9e_fc8a_4612_87b4_e869b304199b.slice/crio-337c10c19e20edff114a3966c9e56035bc39b9682bbdff97b0a2e730cd09b6e7 WatchSource:0}: Error finding container 337c10c19e20edff114a3966c9e56035bc39b9682bbdff97b0a2e730cd09b6e7: Status 404 returned error can't find the container with id 337c10c19e20edff114a3966c9e56035bc39b9682bbdff97b0a2e730cd09b6e7 Jul 21 12:29:07.408178 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:07.408131 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" event={"ID":"e73ddc9e-fc8a-4612-87b4-e869b304199b","Type":"ContainerStarted","Data":"337c10c19e20edff114a3966c9e56035bc39b9682bbdff97b0a2e730cd09b6e7"} Jul 21 12:29:07.635868 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:07.635821 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:29:07.636066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:07.635933 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:29:07.642871 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:07.642461 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:29:08.416438 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:08.416399 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:29:08.462010 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:08.461976 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:29:08.561783 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:08.561743 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:08.627751 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:08.627702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:08.685580 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:08.685535 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:09.418739 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.418639 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" event={"ID":"e73ddc9e-fc8a-4612-87b4-e869b304199b","Type":"ContainerStarted","Data":"1beba294afc949c48e1e881633a1dfb8d25824a57e930575a83ab88327c18a8a"} Jul 21 12:29:09.418739 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.418714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" event={"ID":"e73ddc9e-fc8a-4612-87b4-e869b304199b","Type":"ContainerStarted","Data":"42c0872b22f5c79eac30a417b437e1b0be317ff62d99c14e14dff62402d5e312"} Jul 21 12:29:09.440806 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.440739 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-8p6jf" podStartSLOduration=1.47635864 podStartE2EDuration="3.440723694s" totalImagesPullingTime="1.964365054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:09.439865472 +0000 UTC m=+96.962593189" watchObservedRunningTime="2026-07-21 12:29:09.440723694 +0000 UTC m=+96.963451409" Jul 21 12:29:09.570228 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.570149 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:09.785011 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.784969 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:09.885713 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.885670 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:09.987123 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:09.987082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:10.383365 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:10.383326 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:29:10.475066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:10.475025 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:10.573675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:10.573641 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:11.495009 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.494976 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:11.661496 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.661454 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:11.735819 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.735774 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9"] Jul 21 12:29:11.789722 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.789619 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:11.796798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.796765 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9"] Jul 21 12:29:11.796965 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.796816 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7822s"] Jul 21 12:29:11.796965 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.796871 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.799778 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.799752 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 12:29:11.799778 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.799778 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tqn4j\"" Jul 21 12:29:11.799976 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.799841 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 12:29:11.816935 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.816905 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pdhc2"] Jul 21 12:29:11.817105 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.817042 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.819520 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.819496 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pbx5s\"" Jul 21 12:29:11.819660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.819531 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 21 12:29:11.819660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.819547 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 21 12:29:11.819660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.819596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 21 12:29:11.835042 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.835005 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7822s"] Jul 21 12:29:11.835199 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.835147 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.838008 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.837880 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjw6b\"" Jul 21 12:29:11.838008 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.837909 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 12:29:11.838165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.838146 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 12:29:11.838392 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.838374 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 12:29:11.868177 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868144 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868182 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868241 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868290 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-metrics-client-ca\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868320 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.868391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868376 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868409 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-wtmp\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868433 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-sys\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868523 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7nh8\" (UniqueName: \"kubernetes.io/projected/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-kube-api-access-q7nh8\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868606 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868634 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh49g\" (UniqueName: \"kubernetes.io/projected/6866ac58-136c-4302-8088-7e911861c718-kube-api-access-wh49g\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.868680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-textfile\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-root\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868733 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6866ac58-136c-4302-8088-7e911861c718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.868999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868764 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.868999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868778 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brccb\" (UniqueName: \"kubernetes.io/projected/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-kube-api-access-brccb\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.868999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.868807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.886915 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.886879 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:11.969623 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969571 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-wtmp\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.969811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969634 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-sys\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.969811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.969811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969700 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7nh8\" (UniqueName: \"kubernetes.io/projected/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-kube-api-access-q7nh8\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.969811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-wtmp\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969777 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-sys\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.969823 2584 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969828 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.969892 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls podName:6866ac58-136c-4302-8088-7e911861c718 nodeName:}" failed. No retries permitted until 2026-07-21 12:29:12.469869662 +0000 UTC m=+99.992597362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-7822s" (UID: "6866ac58-136c-4302-8088-7e911861c718") : secret "kube-state-metrics-tls" not found Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969930 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh49g\" (UniqueName: \"kubernetes.io/projected/6866ac58-136c-4302-8088-7e911861c718-kube-api-access-wh49g\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969957 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-textfile\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.969995 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-root\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970030 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6866ac58-136c-4302-8088-7e911861c718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970090 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970116 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brccb\" (UniqueName: \"kubernetes.io/projected/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-kube-api-access-brccb\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970123 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-root\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970155 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970271 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970301 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970402 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-metrics-client-ca\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970456 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.970617 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970633 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.970674 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls podName:3a37cd6e-1ca8-415c-bfbb-7048bf5643fd nodeName:}" failed. No retries permitted until 2026-07-21 12:29:12.470652581 +0000 UTC m=+99.993380289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls") pod "node-exporter-pdhc2" (UID: "3a37cd6e-1ca8-415c-bfbb-7048bf5643fd") : secret "node-exporter-tls" not found Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.970734 2584 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:11.970789 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls podName:629adf2e-3a9b-4c51-bb75-dcaebc1c1156 nodeName:}" failed. No retries permitted until 2026-07-21 12:29:12.470770931 +0000 UTC m=+99.993498639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-6mqs9" (UID: "629adf2e-3a9b-4c51-bb75-dcaebc1c1156") : secret "openshift-state-metrics-tls" not found Jul 21 12:29:11.970945 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.970885 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.971311 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.971181 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.971875 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.971850 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6866ac58-136c-4302-8088-7e911861c718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.972067 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.971953 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-textfile\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.972392 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.972338 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.972887 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.972866 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-metrics-client-ca\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.973497 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.973471 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.974324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.974304 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.974402 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.974355 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.980436 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.980408 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh49g\" (UniqueName: \"kubernetes.io/projected/6866ac58-136c-4302-8088-7e911861c718-kube-api-access-wh49g\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:11.986011 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.985984 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brccb\" (UniqueName: \"kubernetes.io/projected/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-kube-api-access-brccb\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:11.986392 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.986364 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7nh8\" (UniqueName: \"kubernetes.io/projected/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-kube-api-access-q7nh8\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:11.990471 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:11.990445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.093385 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.093298 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.194540 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.194500 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.300752 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.300708 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.395445 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.395351 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.475034 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.474987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:12.475258 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.475070 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:12.475258 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.475153 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:12.477952 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.477916 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a37cd6e-1ca8-415c-bfbb-7048bf5643fd-node-exporter-tls\") pod \"node-exporter-pdhc2\" (UID: \"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd\") " pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:12.478091 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.477969 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/629adf2e-3a9b-4c51-bb75-dcaebc1c1156-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6mqs9\" (UID: \"629adf2e-3a9b-4c51-bb75-dcaebc1c1156\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:12.478091 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.477980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6866ac58-136c-4302-8088-7e911861c718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7822s\" (UID: \"6866ac58-136c-4302-8088-7e911861c718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:12.708039 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.708002 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" Jul 21 12:29:12.728726 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.728699 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" Jul 21 12:29:12.741918 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.741882 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.745828 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.745799 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pdhc2" Jul 21 12:29:12.780082 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:12.780040 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a37cd6e_1ca8_415c_bfbb_7048bf5643fd.slice/crio-f6722e3adccde5ab6a7c6b08d5d064842e3d94084662c9e5eff89fb98f3ab014 WatchSource:0}: Error finding container f6722e3adccde5ab6a7c6b08d5d064842e3d94084662c9e5eff89fb98f3ab014: Status 404 returned error can't find the container with id f6722e3adccde5ab6a7c6b08d5d064842e3d94084662c9e5eff89fb98f3ab014 Jul 21 12:29:12.826523 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.826240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:12.827794 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.827611 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 12:29:12.846074 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.845972 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.850557 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.850444 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 12:29:12.851785 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.851259 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 12:29:12.851785 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.851575 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 12:29:12.852428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.852200 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 12:29:12.856595 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.852860 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 12:29:12.857237 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.856822 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 12:29:12.857237 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.857052 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 12:29:12.858185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.858169 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 12:29:12.858638 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.858615 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jzkn8\"" Jul 21 12:29:12.859066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.859031 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 12:29:12.865570 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.865542 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883493 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-web-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883536 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883597 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883619 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883720 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jdwm\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-kube-api-access-5jdwm\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883764 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883848 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-out\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.884110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.883916 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.916669 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.915522 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9"] Jul 21 12:29:12.920967 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:12.920926 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629adf2e_3a9b_4c51_bb75_dcaebc1c1156.slice/crio-6e2bea5178565280dfa272303fbcfe53e10af998b797f75965c9e35812acbaab WatchSource:0}: Error finding container 6e2bea5178565280dfa272303fbcfe53e10af998b797f75965c9e35812acbaab: Status 404 returned error can't find the container with id 6e2bea5178565280dfa272303fbcfe53e10af998b797f75965c9e35812acbaab Jul 21 12:29:12.944771 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.944732 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7822s"] Jul 21 12:29:12.948024 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:12.947989 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6866ac58_136c_4302_8088_7e911861c718.slice/crio-6a4e5a92072045f54522336149bd111ac98385026c26355c91c95244e000e177 WatchSource:0}: Error finding container 6a4e5a92072045f54522336149bd111ac98385026c26355c91c95244e000e177: Status 404 returned error can't find the container with id 6a4e5a92072045f54522336149bd111ac98385026c26355c91c95244e000e177 Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984437 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984505 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984562 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984631 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984672 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jdwm\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-kube-api-access-5jdwm\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984790 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-out\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984818 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984853 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.985278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.984882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-web-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.988304 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.987162 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.988304 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:12.987321 2584 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 21 12:29:12.988304 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:12.987392 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls podName:fe05a9ec-638a-4ecc-a849-1750cf15d8cb nodeName:}" failed. No retries permitted until 2026-07-21 12:29:13.487371563 +0000 UTC m=+101.010099259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "fe05a9ec-638a-4ecc-a849-1750cf15d8cb") : secret "alertmanager-main-tls" not found Jul 21 12:29:12.988304 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.987523 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.989110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.989085 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.989725 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.989143 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.989900 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.989639 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.990733 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.990705 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-web-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.992014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.991988 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-config-out\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.992703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.992680 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.993334 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.993056 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.993434 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.993357 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.993860 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.993836 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:12.998345 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:12.998321 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jdwm\" (UniqueName: \"kubernetes.io/projected/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-kube-api-access-5jdwm\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:13.280587 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.280493 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dl2nt" Jul 21 12:29:13.371160 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.371115 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:13.439500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.439456 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdhc2" event={"ID":"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd","Type":"ContainerStarted","Data":"f6722e3adccde5ab6a7c6b08d5d064842e3d94084662c9e5eff89fb98f3ab014"} Jul 21 12:29:13.441859 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.441818 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" event={"ID":"6866ac58-136c-4302-8088-7e911861c718","Type":"ContainerStarted","Data":"6a4e5a92072045f54522336149bd111ac98385026c26355c91c95244e000e177"} Jul 21 12:29:13.445163 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.445131 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" event={"ID":"629adf2e-3a9b-4c51-bb75-dcaebc1c1156","Type":"ContainerStarted","Data":"de2a4f3e2755100ba52a47a6df9afbfe93494ed3d01d3d307dc261f44a898241"} Jul 21 12:29:13.445351 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.445192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" event={"ID":"629adf2e-3a9b-4c51-bb75-dcaebc1c1156","Type":"ContainerStarted","Data":"a64f518f38b0a6ee51510cd54816f294c79f84dd231fc285a56ebe404bc84faf"} Jul 21 12:29:13.445351 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.445242 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" event={"ID":"629adf2e-3a9b-4c51-bb75-dcaebc1c1156","Type":"ContainerStarted","Data":"6e2bea5178565280dfa272303fbcfe53e10af998b797f75965c9e35812acbaab"} Jul 21 12:29:13.489363 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.489321 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:13.492581 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.492554 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe05a9ec-638a-4ecc-a849-1750cf15d8cb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe05a9ec-638a-4ecc-a849-1750cf15d8cb\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:13.770845 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:13.770800 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 12:29:14.317011 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:14.316969 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 12:29:14.660942 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:14.660902 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:14.720749 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:14.720712 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe05a9ec_638a_4ecc_a849_1750cf15d8cb.slice/crio-75d9d30d404898d34eb2b6e2f3f8a2872653224f33a63685c0bddb9c3e46bec7 WatchSource:0}: Error finding container 75d9d30d404898d34eb2b6e2f3f8a2872653224f33a63685c0bddb9c3e46bec7: Status 404 returned error can't find the container with id 75d9d30d404898d34eb2b6e2f3f8a2872653224f33a63685c0bddb9c3e46bec7 Jul 21 12:29:14.731693 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:14.731663 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:14.931779 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:14.931502 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:15.062796 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.062626 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:15.460128 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.460035 2584 generic.go:356] "Generic (PLEG): container finished" podID="3a37cd6e-1ca8-415c-bfbb-7048bf5643fd" containerID="823077f2370d6d03f16a96b02c561bfcee169a8fb8957029b1c45cb66c925e84" exitCode=0 Jul 21 12:29:15.460418 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.460342 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdhc2" event={"ID":"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd","Type":"ContainerDied","Data":"823077f2370d6d03f16a96b02c561bfcee169a8fb8957029b1c45cb66c925e84"} Jul 21 12:29:15.460574 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.460419 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:15.462658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.462632 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"75d9d30d404898d34eb2b6e2f3f8a2872653224f33a63685c0bddb9c3e46bec7"} Jul 21 12:29:15.465761 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.465731 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" event={"ID":"6866ac58-136c-4302-8088-7e911861c718","Type":"ContainerStarted","Data":"b5498d2ae14fd768ede5eff608bed87f0422f5a938265984ee20069592d5d84d"} Jul 21 12:29:15.465897 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.465793 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" event={"ID":"6866ac58-136c-4302-8088-7e911861c718","Type":"ContainerStarted","Data":"c66a4dd50d83544767c1f0eae66ca4a66c945321d84b2f9a50b662fc2012b5b8"} Jul 21 12:29:15.465897 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.465822 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" event={"ID":"6866ac58-136c-4302-8088-7e911861c718","Type":"ContainerStarted","Data":"d990b14cb16c86fccfde49bd2f152ed6b1e0a096e41fe121fa7576893d5d71f4"} Jul 21 12:29:15.501107 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.501046 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-7822s" podStartSLOduration=2.555527198 podStartE2EDuration="4.501025491s" totalImagesPullingTime="1.945498293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:15.498763649 +0000 UTC m=+103.021491377" watchObservedRunningTime="2026-07-21 12:29:15.501025491 +0000 UTC m=+103.023753207" Jul 21 12:29:15.557821 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:15.557775 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:16.135962 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.135930 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5ff668846c-jghsb"] Jul 21 12:29:16.145622 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.145572 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148040 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148062 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148268 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148359 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d5irjhci0mupk\"" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148502 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bsbdr\"" Jul 21 12:29:16.148658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.148505 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 21 12:29:16.153321 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.153292 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5ff668846c-jghsb"] Jul 21 12:29:16.216259 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216205 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-client-certs\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216277 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-metrics-server-audit-profiles\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216362 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5e859268-78a0-464e-b08f-7f167f09ccd9-audit-log\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216410 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216440 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px9xt\" (UniqueName: \"kubernetes.io/projected/5e859268-78a0-464e-b08f-7f167f09ccd9-kube-api-access-px9xt\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216469 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-tls\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.216767 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.216494 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-client-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317604 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317502 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5e859268-78a0-464e-b08f-7f167f09ccd9-audit-log\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317604 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317604 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317593 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9xt\" (UniqueName: \"kubernetes.io/projected/5e859268-78a0-464e-b08f-7f167f09ccd9-kube-api-access-px9xt\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317906 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317610 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-tls\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317906 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-client-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317906 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-client-certs\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.317906 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-metrics-server-audit-profiles\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.318098 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.317968 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5e859268-78a0-464e-b08f-7f167f09ccd9-audit-log\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.318567 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.318532 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.318703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.318646 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5e859268-78a0-464e-b08f-7f167f09ccd9-metrics-server-audit-profiles\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.320319 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.320289 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-client-ca-bundle\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.320436 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.320355 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-tls\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.320436 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.320371 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5e859268-78a0-464e-b08f-7f167f09ccd9-secret-metrics-server-client-certs\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.325761 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.325726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9xt\" (UniqueName: \"kubernetes.io/projected/5e859268-78a0-464e-b08f-7f167f09ccd9-kube-api-access-px9xt\") pod \"metrics-server-5ff668846c-jghsb\" (UID: \"5e859268-78a0-464e-b08f-7f167f09ccd9\") " pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.458919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.458870 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:16.472287 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.472251 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" event={"ID":"629adf2e-3a9b-4c51-bb75-dcaebc1c1156","Type":"ContainerStarted","Data":"a4ac5d3b78b5d90771458663a8a777deca7052335520497174dbffdf1de11ac2"} Jul 21 12:29:16.474765 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.474718 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdhc2" event={"ID":"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd","Type":"ContainerStarted","Data":"0382076e0beffb1f0f75b60474a6fdd584b7a7baf5a977e1a2b441465e372917"} Jul 21 12:29:16.474879 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.474797 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdhc2" event={"ID":"3a37cd6e-1ca8-415c-bfbb-7048bf5643fd","Type":"ContainerStarted","Data":"c3b0e3604cae8c74ff56760167722649f8288258294c3b222b3358b198e45d27"} Jul 21 12:29:16.493931 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.493863 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6mqs9" podStartSLOduration=2.706694182 podStartE2EDuration="5.493838702s" totalImagesPullingTime="2.78714452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:16.491979045 +0000 UTC m=+104.014706764" watchObservedRunningTime="2026-07-21 12:29:16.493838702 +0000 UTC m=+104.016566419" Jul 21 12:29:16.518155 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.518077 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pdhc2" podStartSLOduration=3.282552824 podStartE2EDuration="5.518054893s" totalImagesPullingTime="2.110800577s" totalInitContainerRuntime="124.701492ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:16.517426984 +0000 UTC m=+104.040154740" watchObservedRunningTime="2026-07-21 12:29:16.518054893 +0000 UTC m=+104.040782610" Jul 21 12:29:16.541616 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.541548 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:16.562104 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.562075 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:16.589449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.588778 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.603197 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.603119 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:16.621192 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621150 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621386 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621327 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621386 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621381 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621508 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621403 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621508 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621429 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621776 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621750 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.621919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.621848 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcxk7\" (UniqueName: \"kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.645285 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.645255 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5ff668846c-jghsb"] Jul 21 12:29:16.658472 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.658443 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:16.722182 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:16.722430 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722430 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722367 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722430 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722430 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722431 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722501 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722527 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcxk7\" (UniqueName: \"kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.722703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.722567 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.723490 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.723461 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.724111 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.724057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.724824 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.724779 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.725573 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.725528 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.726174 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.726149 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.726423 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.726399 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.732176 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.732149 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcxk7\" (UniqueName: \"kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7\") pod \"console-868b86cf4-grrvk\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:16.844122 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:16.844041 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e859268_78a0_464e_b08f_7f167f09ccd9.slice/crio-d6f7d8eea8a563420341e74b9d3fd967559b45b15ddd25878bdeeb1f41b8be9d WatchSource:0}: Error finding container d6f7d8eea8a563420341e74b9d3fd967559b45b15ddd25878bdeeb1f41b8be9d: Status 404 returned error can't find the container with id d6f7d8eea8a563420341e74b9d3fd967559b45b15ddd25878bdeeb1f41b8be9d Jul 21 12:29:16.908428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:16.908385 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:17.055780 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.055748 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:17.188912 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:17.188876 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b832c1f_8c3d_4b8e_ae6e_8b01e90eb1ea.slice/crio-2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19 WatchSource:0}: Error finding container 2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19: Status 404 returned error can't find the container with id 2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19 Jul 21 12:29:17.484963 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.484860 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" event={"ID":"5e859268-78a0-464e-b08f-7f167f09ccd9","Type":"ContainerStarted","Data":"d6f7d8eea8a563420341e74b9d3fd967559b45b15ddd25878bdeeb1f41b8be9d"} Jul 21 12:29:17.486371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.486343 2584 generic.go:356] "Generic (PLEG): container finished" podID="fe05a9ec-638a-4ecc-a849-1750cf15d8cb" containerID="79fc12c0f449b111261adacfba23310e536827a198d6706cb0c8d025276b3174" exitCode=0 Jul 21 12:29:17.486519 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.486430 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerDied","Data":"79fc12c0f449b111261adacfba23310e536827a198d6706cb0c8d025276b3174"} Jul 21 12:29:17.488313 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.488139 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868b86cf4-grrvk" event={"ID":"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea","Type":"ContainerStarted","Data":"24707052377ff18221e74d69ff71404c0cb80ed6f5ff358c23d6e20da04e9575"} Jul 21 12:29:17.488313 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.488177 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868b86cf4-grrvk" event={"ID":"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea","Type":"ContainerStarted","Data":"2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19"} Jul 21 12:29:17.533185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.533131 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-868b86cf4-grrvk" podStartSLOduration=1.533114738 podStartE2EDuration="1.533114738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:17.532312813 +0000 UTC m=+105.055040529" watchObservedRunningTime="2026-07-21 12:29:17.533114738 +0000 UTC m=+105.055842454" Jul 21 12:29:17.961046 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.960983 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:29:17.970419 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.970337 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:18.000660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.998357 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:29:18.000660 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:17.998542 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.003287 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.003257 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 12:29:18.003792 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.003733 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004088 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004114 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004233 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004338 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004340 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 12:29:18.004543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004465 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 12:29:18.004919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004610 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 12:29:18.004919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004831 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 12:29:18.004919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.004847 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 12:29:18.005277 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.005077 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xjsgg\"" Jul 21 12:29:18.005277 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.005183 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 12:29:18.005277 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.005237 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3h6vlnjevqma\"" Jul 21 12:29:18.006956 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.006931 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 12:29:18.013599 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.011596 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 12:29:18.037110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037079 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037110 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037112 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-web-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037400 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037152 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037400 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037400 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037307 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037400 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037345 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037400 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037386 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037417 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037504 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037538 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgbs9\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-kube-api-access-pgbs9\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037573 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037637 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-config-out\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.037664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037662 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037713 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037791 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037878 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037902 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.038006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.037942 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.126151 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.126104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:18.139274 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139401 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139435 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-web-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139479 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139550 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.139698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139648 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140013 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139686 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140013 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140013 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139936 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgbs9\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-kube-api-access-pgbs9\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140013 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.139988 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140029 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-config-out\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140148 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140178 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140236 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140196 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140536 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140269 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.140536 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.141259 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.141259 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.140999 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.141752 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.141530 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.142263 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.142204 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.144379 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.144342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.144558 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.144390 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.144796 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.144780 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.145889 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.145855 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146049 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146022 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4bd5500e-1af2-4448-8754-3da220096de0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146221 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146169 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-web-config\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146221 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146175 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146704 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146398 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146704 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146328 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.146704 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.146649 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd5500e-1af2-4448-8754-3da220096de0-config-out\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.147698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.147648 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.148624 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.148575 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.149512 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.149487 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd5500e-1af2-4448-8754-3da220096de0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.151096 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.151075 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgbs9\" (UniqueName: \"kubernetes.io/projected/4bd5500e-1af2-4448-8754-3da220096de0-kube-api-access-pgbs9\") pod \"prometheus-k8s-0\" (UID: \"4bd5500e-1af2-4448-8754-3da220096de0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.226542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.226434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:18.315685 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.315512 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:18.933273 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:18.933195 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:29:18.935573 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:18.935540 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd5500e_1af2_4448_8754_3da220096de0.slice/crio-60a899451cadff3d2edaac847997abcc7937202ef0609b4fab35fbbca30b1f49 WatchSource:0}: Error finding container 60a899451cadff3d2edaac847997abcc7937202ef0609b4fab35fbbca30b1f49: Status 404 returned error can't find the container with id 60a899451cadff3d2edaac847997abcc7937202ef0609b4fab35fbbca30b1f49 Jul 21 12:29:19.500588 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:19.500548 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"60a899451cadff3d2edaac847997abcc7937202ef0609b4fab35fbbca30b1f49"} Jul 21 12:29:20.505859 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.505761 2584 generic.go:356] "Generic (PLEG): container finished" podID="4bd5500e-1af2-4448-8754-3da220096de0" containerID="6aea5597aa08cea2f77d75fbee6d73b649b47c93da75db50effa6ae4ebe74505" exitCode=0 Jul 21 12:29:20.506290 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.505846 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerDied","Data":"6aea5597aa08cea2f77d75fbee6d73b649b47c93da75db50effa6ae4ebe74505"} Jul 21 12:29:20.507753 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.507693 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" event={"ID":"5e859268-78a0-464e-b08f-7f167f09ccd9","Type":"ContainerStarted","Data":"fcf3350121eceb7b737159bd651d2d76bd6cb16c1d5ddf0e7a7c7998e16739f0"} Jul 21 12:29:20.510289 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.510268 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"7df1771b9dfaf85889bbf58c714b2d4298d83d58f7137bc5958bc14f0fb30930"} Jul 21 12:29:20.510371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.510322 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"452dcfc6621e83bef0b453a31bea4fccc64487879c8674924cbe4fc06c25e469"} Jul 21 12:29:20.510371 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.510348 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"46e273fc49c0cc3c6be6fb6a70ba1cc1988b0d1a2ece42e41910a650fcac091f"} Jul 21 12:29:20.553678 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:20.553623 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" podStartSLOduration=1.798687688 podStartE2EDuration="4.553604093s" totalImagesPullingTime="2.754916405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:20.552427997 +0000 UTC m=+108.075155713" watchObservedRunningTime="2026-07-21 12:29:20.553604093 +0000 UTC m=+108.076331809" Jul 21 12:29:21.518014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:21.517975 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"a81a17e180c02cadf2ddff7d31eb27ddb1dab99f3cc963661626e301177ec329"} Jul 21 12:29:21.518479 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:21.518047 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"05564ea6f2cccb32ff382b9d9f356951d8013d419d0bddef4c7608eb80cf6726"} Jul 21 12:29:22.527953 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:22.527913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe05a9ec-638a-4ecc-a849-1750cf15d8cb","Type":"ContainerStarted","Data":"8dcfee5ea876d3dec19036134e90f2df984eb94eae662d2766b567f496cd0954"} Jul 21 12:29:22.559189 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:22.559117 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.547305307 podStartE2EDuration="10.559098321s" totalImagesPullingTime="5.985819303s" totalInitContainerRuntime="25.973711ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:22.557302739 +0000 UTC m=+110.080030486" watchObservedRunningTime="2026-07-21 12:29:22.559098321 +0000 UTC m=+110.081826015" Jul 21 12:29:23.311505 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.311454 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:23.338990 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.338944 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:29:23.367689 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.367653 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.369111 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.368554 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:29:23.490861 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.490820 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.490885 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6wm\" (UniqueName: \"kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.490959 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491066 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.490988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491293 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.491064 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491293 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.491134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.491293 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.491164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.499144 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.498977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:23.562832 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.562686 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:23.592730 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.592685 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.592935 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.592755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.593259 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.593202 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.593841 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.593507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.593841 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.593679 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6wm\" (UniqueName: \"kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.593884 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.594014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594155 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.594073 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594155 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.594111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594576 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.594547 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.594798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.594770 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.595844 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.595815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.596661 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.596637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.603038 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.603013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6wm\" (UniqueName: \"kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm\") pod \"console-66499cf8dd-wvmhl\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:23.688084 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:23.688046 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:24.014674 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.014644 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:29:24.016986 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:29:24.016959 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4687367_7da3_406e_87c5_9929c1952fdb.slice/crio-feb7d5881e38fc9f767166064a4a24341989e258697257129a4ad8cbdbc29de9 WatchSource:0}: Error finding container feb7d5881e38fc9f767166064a4a24341989e258697257129a4ad8cbdbc29de9: Status 404 returned error can't find the container with id feb7d5881e38fc9f767166064a4a24341989e258697257129a4ad8cbdbc29de9 Jul 21 12:29:24.292744 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.292694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:24.540650 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.540606 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"f387ac348447111371bbe3f38953d20c7cc9445ef450c6b3aac38f403effc2da"} Jul 21 12:29:24.540826 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.540742 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"654cf4e6a2f635668082f5c05a0cd7f32d3c584c7c7b0b896eca5b83101478b9"} Jul 21 12:29:24.542281 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.542245 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66499cf8dd-wvmhl" event={"ID":"c4687367-7da3-406e-87c5-9929c1952fdb","Type":"ContainerStarted","Data":"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0"} Jul 21 12:29:24.542449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.542301 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66499cf8dd-wvmhl" event={"ID":"c4687367-7da3-406e-87c5-9929c1952fdb","Type":"ContainerStarted","Data":"feb7d5881e38fc9f767166064a4a24341989e258697257129a4ad8cbdbc29de9"} Jul 21 12:29:24.563550 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:24.563491 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66499cf8dd-wvmhl" podStartSLOduration=1.563471833 podStartE2EDuration="1.563471833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:24.56153425 +0000 UTC m=+112.084261966" watchObservedRunningTime="2026-07-21 12:29:24.563471833 +0000 UTC m=+112.086199549" Jul 21 12:29:26.908946 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:26.908906 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:28.566910 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:28.566873 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"25f34e75facf64cad08ae86ce29047e578c3159179b8a42dea020b99a6c460ff"} Jul 21 12:29:28.567443 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:28.566943 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"118325d501aa25c9ab842948f83d4f821e45bef5db418705e434fa3c545e13df"} Jul 21 12:29:28.567443 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:28.566970 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"2685d5f89271a90e5db5a5e5ec35a69c4c85b285421488e5eece3de5a5405a4c"} Jul 21 12:29:28.567443 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:28.566997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4bd5500e-1af2-4448-8754-3da220096de0","Type":"ContainerStarted","Data":"1367748d684ca8b6228b294105a639773d9801cf3278953249b84b1635a42f1e"} Jul 21 12:29:28.597908 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:28.597841 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.622640047 podStartE2EDuration="11.59780759s" totalImagesPullingTime="6.95786941s" totalInitContainerRuntime="17.298133ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:28.595502939 +0000 UTC m=+116.118230668" watchObservedRunningTime="2026-07-21 12:29:28.59780759 +0000 UTC m=+116.120535307" Jul 21 12:29:30.875122 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:30.875089 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:30.975553 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:30.975518 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:33.316138 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.316100 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:29:33.488993 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.488956 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d64c9b84-whd8g" podUID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" containerName="console" containerID="cri-o://b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445" gracePeriod=15 Jul 21 12:29:33.688578 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.688552 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:33.688741 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.688610 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:33.693315 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.693286 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:33.773474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.773439 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:29:33.889884 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.889794 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.889884 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.889849 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4jr6\" (UniqueName: \"kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.890087 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.889935 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.890087 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.889997 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.890087 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.890024 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.890087 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.890052 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config\") pod \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\" (UID: \"d21d9076-e82f-437b-98f6-8aa15fc3d92b\") " Jul 21 12:29:33.890347 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.890318 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:33.890421 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.890402 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config" (OuterVolumeSpecName: "console-config") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:33.890482 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.890457 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:33.892307 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.892280 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:33.892411 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.892304 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:33.892457 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.892441 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6" (OuterVolumeSpecName: "kube-api-access-z4jr6") pod "d21d9076-e82f-437b-98f6-8aa15fc3d92b" (UID: "d21d9076-e82f-437b-98f6-8aa15fc3d92b"). InnerVolumeSpecName "kube-api-access-z4jr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:29:33.991788 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991749 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:33.991788 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991782 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4jr6\" (UniqueName: \"kubernetes.io/projected/d21d9076-e82f-437b-98f6-8aa15fc3d92b-kube-api-access-z4jr6\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:33.991788 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991795 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:33.992078 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991805 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:33.992078 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991816 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21d9076-e82f-437b-98f6-8aa15fc3d92b-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:33.992078 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:33.991825 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21d9076-e82f-437b-98f6-8aa15fc3d92b-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:34.593836 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.593803 2584 generic.go:356] "Generic (PLEG): container finished" podID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" containerID="b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445" exitCode=0 Jul 21 12:29:34.594445 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.593868 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64c9b84-whd8g" event={"ID":"d21d9076-e82f-437b-98f6-8aa15fc3d92b","Type":"ContainerDied","Data":"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445"} Jul 21 12:29:34.594445 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.593948 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d64c9b84-whd8g" event={"ID":"d21d9076-e82f-437b-98f6-8aa15fc3d92b","Type":"ContainerDied","Data":"9a515c0ce170e77e051e49cf030ca34f03ceebc68c1c2c9157e89e5afb69f28c"} Jul 21 12:29:34.594445 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.593980 2584 scope.go:117] "RemoveContainer" containerID="b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445" Jul 21 12:29:34.594445 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.593889 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d64c9b84-whd8g" Jul 21 12:29:34.599918 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.599892 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:29:34.604932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.604914 2584 scope.go:117] "RemoveContainer" containerID="b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445" Jul 21 12:29:34.605205 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:29:34.605175 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445\": container with ID starting with b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445 not found: ID does not exist" containerID="b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445" Jul 21 12:29:34.605344 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.605288 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445"} err="failed to get container status \"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445\": rpc error: code = NotFound desc = could not find container \"b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445\": container with ID starting with b9693011764cc16d1cd8aa0c543a9a2229e8d91e37434f625bfcd1cb05603445 not found: ID does not exist" Jul 21 12:29:34.635960 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.635923 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:29:34.639919 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.639884 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d64c9b84-whd8g"] Jul 21 12:29:34.654969 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.654934 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:29:34.760122 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.760082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:34.860925 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.860823 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:34.961720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.961684 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:29:34.964196 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:34.964171 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" path="/var/lib/kubelet/pods/d21d9076-e82f-437b-98f6-8aa15fc3d92b/volumes" Jul 21 12:29:35.408033 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.407993 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" podUID="570997d6-e8ac-40c8-a663-f39a6fa394ed" containerName="registry" containerID="cri-o://b1743db1794ccd28d0d208f2de068ddb051006a1fb7d61ad13cbc822a78ce7bb" gracePeriod=30 Jul 21 12:29:35.602664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.602627 2584 generic.go:356] "Generic (PLEG): container finished" podID="570997d6-e8ac-40c8-a663-f39a6fa394ed" containerID="b1743db1794ccd28d0d208f2de068ddb051006a1fb7d61ad13cbc822a78ce7bb" exitCode=0 Jul 21 12:29:35.603108 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.602778 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" event={"ID":"570997d6-e8ac-40c8-a663-f39a6fa394ed","Type":"ContainerDied","Data":"b1743db1794ccd28d0d208f2de068ddb051006a1fb7d61ad13cbc822a78ce7bb"} Jul 21 12:29:35.692493 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.692459 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:29:35.808785 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808751 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.808973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808818 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2w44\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.808973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808857 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.808973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808899 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.808973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808927 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.808973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808964 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.809268 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.808980 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.809268 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.809003 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates\") pod \"570997d6-e8ac-40c8-a663-f39a6fa394ed\" (UID: \"570997d6-e8ac-40c8-a663-f39a6fa394ed\") " Jul 21 12:29:35.809561 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.809509 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:35.809692 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.809658 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:35.811925 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.811845 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:35.811925 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.811910 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44" (OuterVolumeSpecName: "kube-api-access-r2w44") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "kube-api-access-r2w44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:29:35.812133 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.812112 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:29:35.812227 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.812127 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:35.812286 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.812203 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:29:35.817442 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.817418 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "570997d6-e8ac-40c8-a663-f39a6fa394ed" (UID: "570997d6-e8ac-40c8-a663-f39a6fa394ed"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910532 2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/570997d6-e8ac-40c8-a663-f39a6fa394ed-ca-trust-extracted\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910565 2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-tls\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910576 2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-registry-certificates\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910585 2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-bound-sa-token\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910593 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2w44\" (UniqueName: \"kubernetes.io/projected/570997d6-e8ac-40c8-a663-f39a6fa394ed-kube-api-access-r2w44\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910601 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/570997d6-e8ac-40c8-a663-f39a6fa394ed-trusted-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910611 2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-installation-pull-secrets\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:35.910625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:35.910619 2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/570997d6-e8ac-40c8-a663-f39a6fa394ed-image-registry-private-configuration\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:36.459073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.459039 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:36.459291 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.459129 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:36.608472 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.608444 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" Jul 21 12:29:36.608934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.608487 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fbb7d9bcb-988vt" event={"ID":"570997d6-e8ac-40c8-a663-f39a6fa394ed","Type":"ContainerDied","Data":"f820c36dda7e471f63ba17aa0e642340d431a327760a0b5450747bcb2d8a6ef9"} Jul 21 12:29:36.608934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.608545 2584 scope.go:117] "RemoveContainer" containerID="b1743db1794ccd28d0d208f2de068ddb051006a1fb7d61ad13cbc822a78ce7bb" Jul 21 12:29:36.630999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.630966 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:29:36.637049 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.637017 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fbb7d9bcb-988vt"] Jul 21 12:29:36.967891 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:36.967860 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570997d6-e8ac-40c8-a663-f39a6fa394ed" path="/var/lib/kubelet/pods/570997d6-e8ac-40c8-a663-f39a6fa394ed/volumes" Jul 21 12:29:48.336234 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.336142 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-868b86cf4-grrvk" podUID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" containerName="console" containerID="cri-o://24707052377ff18221e74d69ff71404c0cb80ed6f5ff358c23d6e20da04e9575" gracePeriod=15 Jul 21 12:29:48.673085 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.673051 2584 generic.go:356] "Generic (PLEG): container finished" podID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" containerID="24707052377ff18221e74d69ff71404c0cb80ed6f5ff358c23d6e20da04e9575" exitCode=0 Jul 21 12:29:48.673291 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.673129 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868b86cf4-grrvk" event={"ID":"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea","Type":"ContainerDied","Data":"24707052377ff18221e74d69ff71404c0cb80ed6f5ff358c23d6e20da04e9575"} Jul 21 12:29:48.673291 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.673234 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868b86cf4-grrvk" event={"ID":"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea","Type":"ContainerDied","Data":"2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19"} Jul 21 12:29:48.673291 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.673253 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2181449379fb70e669ff33207efe09343432216c14f4bad05b84e59399fa8e19" Jul 21 12:29:48.673616 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.673603 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:48.834127 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834081 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834143 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834194 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834238 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcxk7\" (UniqueName: \"kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834266 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834337 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.834381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.834375 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert\") pod \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\" (UID: \"4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea\") " Jul 21 12:29:48.835042 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.835006 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:48.835872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.835840 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config" (OuterVolumeSpecName: "console-config") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:48.836111 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.836023 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca" (OuterVolumeSpecName: "service-ca") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:48.836311 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.836284 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:29:48.841980 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.841943 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7" (OuterVolumeSpecName: "kube-api-access-lcxk7") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "kube-api-access-lcxk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:29:48.842408 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.842362 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:48.843907 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.843602 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" (UID: "4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:29:48.935613 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935581 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935613 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935616 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935628 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935637 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935646 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935654 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcxk7\" (UniqueName: \"kubernetes.io/projected/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-kube-api-access-lcxk7\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:48.935833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:48.935665 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:29:49.677865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:49.677827 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868b86cf4-grrvk" Jul 21 12:29:49.694433 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:49.694399 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:49.697879 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:49.697848 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-868b86cf4-grrvk"] Jul 21 12:29:50.964987 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:50.964954 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" path="/var/lib/kubelet/pods/4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea/volumes" Jul 21 12:29:56.464296 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:56.464258 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:56.468530 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:56.468499 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5ff668846c-jghsb" Jul 21 12:29:59.683650 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:59.683598 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fb67d6d95-8kw22" podUID="f726c98e-8e26-4392-bf25-a620b1cd7d79" containerName="console" containerID="cri-o://c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c" gracePeriod=15 Jul 21 12:29:59.932872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:29:59.932848 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:30:00.019562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019457 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019526 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019560 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019852 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019598 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019852 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019622 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl5cr\" (UniqueName: \"kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019852 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019641 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.019852 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019675 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config\") pod \"f726c98e-8e26-4392-bf25-a620b1cd7d79\" (UID: \"f726c98e-8e26-4392-bf25-a620b1cd7d79\") " Jul 21 12:30:00.020050 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.019994 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca" (OuterVolumeSpecName: "service-ca") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:30:00.020050 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.020021 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:30:00.020171 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.020031 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:30:00.020270 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.020246 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config" (OuterVolumeSpecName: "console-config") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:30:00.021780 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.021751 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:30:00.021891 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.021806 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr" (OuterVolumeSpecName: "kube-api-access-tl5cr") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "kube-api-access-tl5cr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:30:00.021891 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.021827 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f726c98e-8e26-4392-bf25-a620b1cd7d79" (UID: "f726c98e-8e26-4392-bf25-a620b1cd7d79"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:30:00.120697 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120658 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120697 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120692 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl5cr\" (UniqueName: \"kubernetes.io/projected/f726c98e-8e26-4392-bf25-a620b1cd7d79-kube-api-access-tl5cr\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120697 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120702 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120711 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120719 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120727 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f726c98e-8e26-4392-bf25-a620b1cd7d79-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.120932 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.120735 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f726c98e-8e26-4392-bf25-a620b1cd7d79-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:30:00.725140 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.725101 2584 generic.go:356] "Generic (PLEG): container finished" podID="f726c98e-8e26-4392-bf25-a620b1cd7d79" containerID="c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c" exitCode=0 Jul 21 12:30:00.725654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.725146 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb67d6d95-8kw22" event={"ID":"f726c98e-8e26-4392-bf25-a620b1cd7d79","Type":"ContainerDied","Data":"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c"} Jul 21 12:30:00.725654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.725156 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb67d6d95-8kw22" Jul 21 12:30:00.725654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.725189 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb67d6d95-8kw22" event={"ID":"f726c98e-8e26-4392-bf25-a620b1cd7d79","Type":"ContainerDied","Data":"2c84445b2548e56c9b40fece7b8a6dccd6fab53abd475a11bf0e38bcb774ec71"} Jul 21 12:30:00.725654 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.725226 2584 scope.go:117] "RemoveContainer" containerID="c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c" Jul 21 12:30:00.733659 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.733642 2584 scope.go:117] "RemoveContainer" containerID="c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c" Jul 21 12:30:00.733923 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:30:00.733905 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c\": container with ID starting with c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c not found: ID does not exist" containerID="c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c" Jul 21 12:30:00.733971 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.733930 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c"} err="failed to get container status \"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c\": rpc error: code = NotFound desc = could not find container \"c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c\": container with ID starting with c67f4d153e734ae08455b9b2a556ef97cf9ea40e512649bab011075a471d258c not found: ID does not exist" Jul 21 12:30:00.747381 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.747351 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:30:00.751500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.751471 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fb67d6d95-8kw22"] Jul 21 12:30:00.964480 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:00.964448 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f726c98e-8e26-4392-bf25-a620b1cd7d79" path="/var/lib/kubelet/pods/f726c98e-8e26-4392-bf25-a620b1cd7d79/volumes" Jul 21 12:30:18.316181 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:18.316144 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:30:18.332602 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:18.332564 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:30:18.824783 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:18.824751 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:30:27.816349 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:27.816250 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:27.872318 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:27.872275 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:39.830602 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:39.830559 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:39.888831 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:39.888781 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:41.829511 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829479 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829801 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f726c98e-8e26-4392-bf25-a620b1cd7d79" containerName="console" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829811 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f726c98e-8e26-4392-bf25-a620b1cd7d79" containerName="console" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829830 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570997d6-e8ac-40c8-a663-f39a6fa394ed" containerName="registry" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829835 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="570997d6-e8ac-40c8-a663-f39a6fa394ed" containerName="registry" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829842 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" containerName="console" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829849 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" containerName="console" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829860 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" containerName="console" Jul 21 12:30:41.829896 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829865 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" containerName="console" Jul 21 12:30:41.830165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829912 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="570997d6-e8ac-40c8-a663-f39a6fa394ed" containerName="registry" Jul 21 12:30:41.830165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829920 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21d9076-e82f-437b-98f6-8aa15fc3d92b" containerName="console" Jul 21 12:30:41.830165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829926 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f726c98e-8e26-4392-bf25-a620b1cd7d79" containerName="console" Jul 21 12:30:41.830165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.829932 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b832c1f-8c3d-4b8e-ae6e-8b01e90eb1ea" containerName="console" Jul 21 12:30:41.832897 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.832875 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.842552 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.842519 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:30:41.879228 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.879180 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:41.895722 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.895901 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895734 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.895901 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895815 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.895901 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895857 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.895901 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46tk\" (UniqueName: \"kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.896053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.895965 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.896053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.896009 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.937756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.937712 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:41.996969 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.996930 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997031 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997050 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997075 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997117 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.997247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.997148 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46tk\" (UniqueName: \"kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.998722 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.998686 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.998899 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.998712 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.999015 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.998731 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:41.999103 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:41.998814 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:42.000934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.000905 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:42.002074 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.002050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:42.007315 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.007289 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46tk\" (UniqueName: \"kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk\") pod \"console-7976b7bd5-bv2zt\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:42.142791 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.142689 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:42.276435 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.276408 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:30:42.278814 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:30:42.278784 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb86c6ed_eacd_45d2_884d_80a0be574d63.slice/crio-9755f67577b518353595e8621d914d3b6e490036646382c177fe77d47f559f80 WatchSource:0}: Error finding container 9755f67577b518353595e8621d914d3b6e490036646382c177fe77d47f559f80: Status 404 returned error can't find the container with id 9755f67577b518353595e8621d914d3b6e490036646382c177fe77d47f559f80 Jul 21 12:30:42.911696 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.911653 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7976b7bd5-bv2zt" event={"ID":"fb86c6ed-eacd-45d2-884d-80a0be574d63","Type":"ContainerStarted","Data":"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f"} Jul 21 12:30:42.912174 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.911717 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7976b7bd5-bv2zt" event={"ID":"fb86c6ed-eacd-45d2-884d-80a0be574d63","Type":"ContainerStarted","Data":"9755f67577b518353595e8621d914d3b6e490036646382c177fe77d47f559f80"} Jul 21 12:30:42.929724 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:42.929661 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7976b7bd5-bv2zt" podStartSLOduration=1.929642645 podStartE2EDuration="1.929642645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:30:42.928521578 +0000 UTC m=+190.451249297" watchObservedRunningTime="2026-07-21 12:30:42.929642645 +0000 UTC m=+190.452370360" Jul 21 12:30:45.356441 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:45.356400 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:45.469367 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:45.469322 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:45.569341 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:45.569302 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:52.143737 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:52.143702 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:52.144136 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:52.143783 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:52.148556 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:52.148534 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:52.965353 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:52.965317 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:30:53.010395 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:53.010359 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:30:53.126024 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:53.125988 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:53.226208 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:53.226115 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:30:57.712653 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:30:57.712619 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:05.314801 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:05.314764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:05.370885 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:05.370843 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:18.035396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.035348 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66499cf8dd-wvmhl" podUID="c4687367-7da3-406e-87c5-9929c1952fdb" containerName="console" containerID="cri-o://e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0" gracePeriod=15 Jul 21 12:31:18.286956 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.286888 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:31:18.427341 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427303 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427358 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427405 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht6wm\" (UniqueName: \"kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427432 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427460 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427885 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427546 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.427885 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427630 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert\") pod \"c4687367-7da3-406e-87c5-9929c1952fdb\" (UID: \"c4687367-7da3-406e-87c5-9929c1952fdb\") " Jul 21 12:31:18.428014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427843 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config" (OuterVolumeSpecName: "console-config") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:31:18.428014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427975 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:31:18.428014 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.427985 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:31:18.428157 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.428121 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:31:18.429747 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.429721 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:31:18.429865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.429777 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm" (OuterVolumeSpecName: "kube-api-access-ht6wm") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "kube-api-access-ht6wm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:31:18.429865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.429825 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4687367-7da3-406e-87c5-9929c1952fdb" (UID: "c4687367-7da3-406e-87c5-9929c1952fdb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:31:18.529053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529004 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529050 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529062 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht6wm\" (UniqueName: \"kubernetes.io/projected/c4687367-7da3-406e-87c5-9929c1952fdb-kube-api-access-ht6wm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529071 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529367 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529080 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529367 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529088 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4687367-7da3-406e-87c5-9929c1952fdb-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:18.529367 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:18.529097 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4687367-7da3-406e-87c5-9929c1952fdb-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:31:19.068112 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.068075 2584 generic.go:356] "Generic (PLEG): container finished" podID="c4687367-7da3-406e-87c5-9929c1952fdb" containerID="e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0" exitCode=0 Jul 21 12:31:19.068621 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.068127 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66499cf8dd-wvmhl" Jul 21 12:31:19.068621 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.068163 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66499cf8dd-wvmhl" event={"ID":"c4687367-7da3-406e-87c5-9929c1952fdb","Type":"ContainerDied","Data":"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0"} Jul 21 12:31:19.068621 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.068231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66499cf8dd-wvmhl" event={"ID":"c4687367-7da3-406e-87c5-9929c1952fdb","Type":"ContainerDied","Data":"feb7d5881e38fc9f767166064a4a24341989e258697257129a4ad8cbdbc29de9"} Jul 21 12:31:19.068621 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.068252 2584 scope.go:117] "RemoveContainer" containerID="e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0" Jul 21 12:31:19.076849 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.076829 2584 scope.go:117] "RemoveContainer" containerID="e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0" Jul 21 12:31:19.077125 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:31:19.077107 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0\": container with ID starting with e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0 not found: ID does not exist" containerID="e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0" Jul 21 12:31:19.077171 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.077134 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0"} err="failed to get container status \"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0\": rpc error: code = NotFound desc = could not find container \"e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0\": container with ID starting with e8506b07a8b31c84daae3357417eb12929f21b219bd155cc9cc7bace188974c0 not found: ID does not exist" Jul 21 12:31:19.087342 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.087309 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:31:19.090284 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:19.090255 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66499cf8dd-wvmhl"] Jul 21 12:31:20.587525 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:20.587476 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:20.963614 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:20.963581 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4687367-7da3-406e-87c5-9929c1952fdb" path="/var/lib/kubelet/pods/c4687367-7da3-406e-87c5-9929c1952fdb/volumes" Jul 21 12:31:28.178302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:28.178257 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:28.275067 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:28.275026 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:50.203797 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:50.203736 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:51.036805 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:51.036766 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:53.815728 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:53.815696 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:53.879938 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:53.879897 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:55.401225 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:55.401117 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:55.512598 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:55.512546 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:55.769933 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:55.769878 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:57.790799 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:57.790764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:31:58.052294 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:31:58.052141 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:00.419665 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:00.419631 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:00.520954 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:00.520918 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:04.978728 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:04.978692 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:05.736691 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.736651 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qvqs"] Jul 21 12:32:05.737024 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.737011 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4687367-7da3-406e-87c5-9929c1952fdb" containerName="console" Jul 21 12:32:05.737024 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.737025 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4687367-7da3-406e-87c5-9929c1952fdb" containerName="console" Jul 21 12:32:05.737103 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.737078 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4687367-7da3-406e-87c5-9929c1952fdb" containerName="console" Jul 21 12:32:05.739034 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.738997 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.742500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.742473 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 12:32:05.742500 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.742497 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 12:32:05.743243 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.743226 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-86vb7\"" Jul 21 12:32:05.751949 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.751917 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qvqs"] Jul 21 12:32:05.754599 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.754565 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.754733 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.754629 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lkv\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-kube-api-access-56lkv\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.855272 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.855194 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.855486 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.855284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lkv\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-kube-api-access-56lkv\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.866326 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.866284 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:05.866501 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:05.866480 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lkv\" (UniqueName: \"kubernetes.io/projected/ae409c5b-096e-4f4d-81f3-2f8479af29a0-kube-api-access-56lkv\") pod \"cert-manager-55b4d954d5-9qvqs\" (UID: \"ae409c5b-096e-4f4d-81f3-2f8479af29a0\") " pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:06.057180 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:06.057067 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9qvqs" Jul 21 12:32:06.208323 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:06.208294 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qvqs"] Jul 21 12:32:06.211103 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:32:06.211071 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae409c5b_096e_4f4d_81f3_2f8479af29a0.slice/crio-8a73c1a80e0e3887d3079b664b4dd6edbaf165fe56c977c8e6540b9bbd251fb6 WatchSource:0}: Error finding container 8a73c1a80e0e3887d3079b664b4dd6edbaf165fe56c977c8e6540b9bbd251fb6: Status 404 returned error can't find the container with id 8a73c1a80e0e3887d3079b664b4dd6edbaf165fe56c977c8e6540b9bbd251fb6 Jul 21 12:32:06.264097 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:06.264056 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9qvqs" event={"ID":"ae409c5b-096e-4f4d-81f3-2f8479af29a0","Type":"ContainerStarted","Data":"8a73c1a80e0e3887d3079b664b4dd6edbaf165fe56c977c8e6540b9bbd251fb6"} Jul 21 12:32:07.828449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:07.828387 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:07.927518 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:07.927440 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:10.284585 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:10.284541 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9qvqs" event={"ID":"ae409c5b-096e-4f4d-81f3-2f8479af29a0","Type":"ContainerStarted","Data":"b1f2b1eaf8476763ff409b647db60c0a9bdb94895de09f636bba8ffe63843520"} Jul 21 12:32:10.317913 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:10.317856 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9qvqs" podStartSLOduration=2.244132641 podStartE2EDuration="5.317835896s" totalImagesPullingTime="3.073703255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:10.315719023 +0000 UTC m=+277.838446739" watchObservedRunningTime="2026-07-21 12:32:10.317835896 +0000 UTC m=+277.840563612" Jul 21 12:32:13.702468 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:13.702426 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:13.810562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:13.810522 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:15.357911 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:15.357871 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:23.686370 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:23.686332 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:27.488087 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:27.488051 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:27.588100 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:27.588064 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.112468 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.112414 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.428052 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.428014 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.540992 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.540957 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.560709 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.560669 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt"] Jul 21 12:32:28.563112 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.563091 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.570232 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.566472 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 12:32:28.570232 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.566869 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 12:32:28.570232 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.566895 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 12:32:28.570232 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.567061 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 12:32:28.570232 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.567691 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-49kkt\"" Jul 21 12:32:28.592874 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.592839 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt"] Jul 21 12:32:28.663082 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.663034 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.663325 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.663089 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvsn\" (UniqueName: \"kubernetes.io/projected/4a6e2033-527e-4719-9ab8-e9576135c34e-kube-api-access-ldvsn\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.663325 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.663151 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.663325 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.663248 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.663446 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.663352 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.711391 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.711293 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.764389 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.764603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764415 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.764603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.764603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvsn\" (UniqueName: \"kubernetes.io/projected/4a6e2033-527e-4719-9ab8-e9576135c34e-kube-api-access-ldvsn\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.764603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764520 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.764969 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.764947 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.766904 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.766878 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6e2033-527e-4719-9ab8-e9576135c34e-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.767056 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.767041 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.767185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.767165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a6e2033-527e-4719-9ab8-e9576135c34e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.774700 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.774668 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvsn\" (UniqueName: \"kubernetes.io/projected/4a6e2033-527e-4719-9ab8-e9576135c34e-kube-api-access-ldvsn\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-hkrlt\" (UID: \"4a6e2033-527e-4719-9ab8-e9576135c34e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.813247 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.813187 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:28.879295 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.879260 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:28.911523 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:28.911467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:29.021885 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:29.021853 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:29.045290 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:29.045254 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt"] Jul 21 12:32:29.049050 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:32:29.049021 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6e2033_527e_4719_9ab8_e9576135c34e.slice/crio-dc320ab9cea65b57bb6075280432391fd5a43bfa5635af973c8092ce047bea72 WatchSource:0}: Error finding container dc320ab9cea65b57bb6075280432391fd5a43bfa5635af973c8092ce047bea72: Status 404 returned error can't find the container with id dc320ab9cea65b57bb6075280432391fd5a43bfa5635af973c8092ce047bea72 Jul 21 12:32:29.147161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:29.147115 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:29.248724 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:29.248632 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:29.365367 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:29.365329 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" event={"ID":"4a6e2033-527e-4719-9ab8-e9576135c34e","Type":"ContainerStarted","Data":"dc320ab9cea65b57bb6075280432391fd5a43bfa5635af973c8092ce047bea72"} Jul 21 12:32:31.717499 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:31.717448 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:31.831080 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:31.831037 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:32.383562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:32.383517 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" event={"ID":"4a6e2033-527e-4719-9ab8-e9576135c34e","Type":"ContainerStarted","Data":"2445ba408f0276c68d7fa006b9d4e15ad2148c713f459232a48e6e28fe3a4f4b"} Jul 21 12:32:32.832384 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:32.832354 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:32:32.837491 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:32.837457 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:32:32.842244 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:32.841145 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 12:32:33.390207 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:33.390169 2584 generic.go:356] "Generic (PLEG): container finished" podID="4a6e2033-527e-4719-9ab8-e9576135c34e" containerID="2445ba408f0276c68d7fa006b9d4e15ad2148c713f459232a48e6e28fe3a4f4b" exitCode=0 Jul 21 12:32:33.390434 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:33.390256 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" event={"ID":"4a6e2033-527e-4719-9ab8-e9576135c34e","Type":"ContainerDied","Data":"2445ba408f0276c68d7fa006b9d4e15ad2148c713f459232a48e6e28fe3a4f4b"} Jul 21 12:32:34.396354 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:34.396319 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" event={"ID":"4a6e2033-527e-4719-9ab8-e9576135c34e","Type":"ContainerStarted","Data":"82a4a13dfe86382f0cb8f6336dbc8cee395cf1ddb137e52fadfaa7b584f40bfb"} Jul 21 12:32:34.396787 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:34.396447 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:34.422682 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:34.422620 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" podStartSLOduration=1.787007671 podStartE2EDuration="6.422599884s" totalImagesPullingTime="3.513423367s" totalInitContainerRuntime="1.122168846s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:34.420317002 +0000 UTC m=+301.943044718" watchObservedRunningTime="2026-07-21 12:32:34.422599884 +0000 UTC m=+301.945327600" Jul 21 12:32:45.403290 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:45.403257 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-hkrlt" Jul 21 12:32:45.879019 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:45.878967 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:46.045432 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.045391 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp"] Jul 21 12:32:46.054023 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.053992 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.062498 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.062468 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 12:32:46.063698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.063669 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 12:32:46.063698 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.063679 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vmscw\"" Jul 21 12:32:46.063884 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.063715 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:32:46.063884 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.063732 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 12:32:46.063884 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.063700 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 12:32:46.092034 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.091986 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp"] Jul 21 12:32:46.225650 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.225583 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66079a3f-aa83-4cd5-a13b-9422ebbcca34-manager-config\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.225872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.225678 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-metrics-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.225872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.225721 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.225872 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.225754 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfq2j\" (UniqueName: \"kubernetes.io/projected/66079a3f-aa83-4cd5-a13b-9422ebbcca34-kube-api-access-pfq2j\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.326649 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.326585 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-metrics-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.326881 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.326685 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.326881 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.326730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq2j\" (UniqueName: \"kubernetes.io/projected/66079a3f-aa83-4cd5-a13b-9422ebbcca34-kube-api-access-pfq2j\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.326881 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.326783 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66079a3f-aa83-4cd5-a13b-9422ebbcca34-manager-config\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.327527 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.327500 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/66079a3f-aa83-4cd5-a13b-9422ebbcca34-manager-config\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.329257 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.329234 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.329329 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.329275 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/66079a3f-aa83-4cd5-a13b-9422ebbcca34-metrics-cert\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.346484 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.346448 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfq2j\" (UniqueName: \"kubernetes.io/projected/66079a3f-aa83-4cd5-a13b-9422ebbcca34-kube-api-access-pfq2j\") pod \"lws-controller-manager-7c45b47d79-rbfsp\" (UID: \"66079a3f-aa83-4cd5-a13b-9422ebbcca34\") " pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.364301 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.364262 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:32:46.510747 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.510720 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp"] Jul 21 12:32:46.513476 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:32:46.513442 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66079a3f_aa83_4cd5_a13b_9422ebbcca34.slice/crio-17eda730d36ec1b3e0f81bc9c849766f1e931de0e839c136d13de3934a0f1f4e WatchSource:0}: Error finding container 17eda730d36ec1b3e0f81bc9c849766f1e931de0e839c136d13de3934a0f1f4e: Status 404 returned error can't find the container with id 17eda730d36ec1b3e0f81bc9c849766f1e931de0e839c136d13de3934a0f1f4e Jul 21 12:32:46.515512 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:46.515494 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:32:47.105731 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:47.105687 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:47.458727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:47.458688 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" event={"ID":"66079a3f-aa83-4cd5-a13b-9422ebbcca34","Type":"ContainerStarted","Data":"17eda730d36ec1b3e0f81bc9c849766f1e931de0e839c136d13de3934a0f1f4e"} Jul 21 12:32:48.125605 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:48.125554 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:48.397618 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:48.397519 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:49.066569 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:49.066525 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:49.226834 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:49.226804 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:49.322918 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:49.322803 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:49.981814 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:49.981774 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:50.173817 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.173782 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:50.550511 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.550468 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:50.696514 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.696480 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2"] Jul 21 12:32:50.699666 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.699644 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.702293 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.702266 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 12:32:50.702811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.702794 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vh25j\"" Jul 21 12:32:50.704946 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.704917 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 12:32:50.711577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.711547 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2"] Jul 21 12:32:50.871369 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.871254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ef682-d12a-4ad9-a603-159f2969bd01-tmp\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.871532 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.871380 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65scg\" (UniqueName: \"kubernetes.io/projected/fd3ef682-d12a-4ad9-a603-159f2969bd01-kube-api-access-65scg\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.871532 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.871432 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3ef682-d12a-4ad9-a603-159f2969bd01-tls-certs\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.972307 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.972266 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ef682-d12a-4ad9-a603-159f2969bd01-tmp\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.972527 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.972336 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65scg\" (UniqueName: \"kubernetes.io/projected/fd3ef682-d12a-4ad9-a603-159f2969bd01-kube-api-access-65scg\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.972527 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.972378 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3ef682-d12a-4ad9-a603-159f2969bd01-tls-certs\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.974860 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.974825 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ef682-d12a-4ad9-a603-159f2969bd01-tmp\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.975070 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.975045 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3ef682-d12a-4ad9-a603-159f2969bd01-tls-certs\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:50.984822 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:50.984794 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65scg\" (UniqueName: \"kubernetes.io/projected/fd3ef682-d12a-4ad9-a603-159f2969bd01-kube-api-access-65scg\") pod \"kube-auth-proxy-5b686cc986-hz7t2\" (UID: \"fd3ef682-d12a-4ad9-a603-159f2969bd01\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:51.011389 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:51.011337 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" Jul 21 12:32:51.020045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:51.020010 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:51.187153 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:51.187111 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2"] Jul 21 12:32:51.190605 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:32:51.190567 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3ef682_d12a_4ad9_a603_159f2969bd01.slice/crio-fd5bd676f7e4b46b5c6fac4ec25d3f1bfc9ba1c759ba50c11a3aba8749954a7d WatchSource:0}: Error finding container fd5bd676f7e4b46b5c6fac4ec25d3f1bfc9ba1c759ba50c11a3aba8749954a7d: Status 404 returned error can't find the container with id fd5bd676f7e4b46b5c6fac4ec25d3f1bfc9ba1c759ba50c11a3aba8749954a7d Jul 21 12:32:51.480625 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:51.480526 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" event={"ID":"fd3ef682-d12a-4ad9-a603-159f2969bd01","Type":"ContainerStarted","Data":"fd5bd676f7e4b46b5c6fac4ec25d3f1bfc9ba1c759ba50c11a3aba8749954a7d"} Jul 21 12:32:51.908141 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:51.908048 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:52.007674 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:52.007635 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:53.255833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:53.255791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:54.498703 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:54.498600 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" event={"ID":"fd3ef682-d12a-4ad9-a603-159f2969bd01","Type":"ContainerStarted","Data":"2b2f5fae0d1a38f9c7e451e594c3e676673df37b95b5ba40f26a96675274dd52"} Jul 21 12:32:54.516496 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:54.516432 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b686cc986-hz7t2" podStartSLOduration=1.490588932 podStartE2EDuration="4.516409091s" totalImagesPullingTime="3.025820159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:54.514773377 +0000 UTC m=+322.037501094" watchObservedRunningTime="2026-07-21 12:32:54.516409091 +0000 UTC m=+322.039136812" Jul 21 12:32:56.547028 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:56.546978 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:56.646587 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:56.646543 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:57.553160 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:57.553113 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:57.762985 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:57.762939 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:32:57.942156 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:32:57.942118 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:00.299734 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:00.299695 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:00.399614 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:00.399496 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:00.560313 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:00.560112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:00.717461 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:00.717426 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:00.822459 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:00.822360 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:01.535865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:01.535823 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" event={"ID":"66079a3f-aa83-4cd5-a13b-9422ebbcca34","Type":"ContainerStarted","Data":"572adfdc98aab99bfb3f0dc811ad22c1bf1a8ef81285918744ae65a204126bcf"} Jul 21 12:33:01.536324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:01.535938 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:33:01.560812 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:01.560750 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" podStartSLOduration=1.67539332 podStartE2EDuration="16.560732492s" totalImagesPullingTime="14.885339172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:01.560463323 +0000 UTC m=+329.083191041" watchObservedRunningTime="2026-07-21 12:33:01.560732492 +0000 UTC m=+329.083460208" Jul 21 12:33:12.543159 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:12.543125 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c45b47d79-rbfsp" Jul 21 12:33:49.663006 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:49.662961 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:49.767619 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:49.767566 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:49.856042 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:49.856006 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:50.606471 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:50.606432 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:33:51.189071 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:33:51.189036 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:07.560501 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:07.560452 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:07.660851 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:07.660812 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:18.944242 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:18.944182 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:19.518459 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:19.518416 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:20.545334 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:20.545294 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:23.804727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:23.804678 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:23.908020 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:23.907984 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:24.233924 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.233887 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b54d755d-knnrx"] Jul 21 12:34:24.237306 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.237275 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.255989 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.255946 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b54d755d-knnrx"] Jul 21 12:34:24.314577 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.314542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:24.414550 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.414506 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-service-ca\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.414957 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.414930 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-trusted-ca-bundle\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.415136 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.415119 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75twt\" (UniqueName: \"kubernetes.io/projected/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-kube-api-access-75twt\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.415277 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.415262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-oauth-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.415416 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.415400 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-oauth-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.415552 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.415536 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.415679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.415664 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.430519 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.430485 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:24.516725 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516618 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-trusted-ca-bundle\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.516725 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75twt\" (UniqueName: \"kubernetes.io/projected/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-kube-api-access-75twt\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.516725 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516680 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-oauth-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.516725 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516699 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-oauth-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517091 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516735 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517091 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516778 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517091 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.516816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-service-ca\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517589 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.517558 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-oauth-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.517637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-service-ca\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.517672 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.517727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.517713 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-trusted-ca-bundle\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.519462 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.519435 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-serving-cert\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.519573 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.519498 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-console-oauth-config\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.533197 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.533151 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75twt\" (UniqueName: \"kubernetes.io/projected/bda00eb1-73a1-4a21-ad07-08567cd1cbcc-kube-api-access-75twt\") pod \"console-76b54d755d-knnrx\" (UID: \"bda00eb1-73a1-4a21-ad07-08567cd1cbcc\") " pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.547903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.547863 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:24.560933 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.560894 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:24.701132 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.701089 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b54d755d-knnrx"] Jul 21 12:34:24.705340 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:34:24.705312 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda00eb1_73a1_4a21_ad07_08567cd1cbcc.slice/crio-934fd10971408a77f9d3ba38170e65f63d7cd5a66fef2b025df40d932368849f WatchSource:0}: Error finding container 934fd10971408a77f9d3ba38170e65f63d7cd5a66fef2b025df40d932368849f: Status 404 returned error can't find the container with id 934fd10971408a77f9d3ba38170e65f63d7cd5a66fef2b025df40d932368849f Jul 21 12:34:24.934333 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.934294 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b54d755d-knnrx" event={"ID":"bda00eb1-73a1-4a21-ad07-08567cd1cbcc","Type":"ContainerStarted","Data":"b7d3c50029a565cc54c24bc23b2b251ee4b75e9bdd70bad384f6b94453dbb8ed"} Jul 21 12:34:24.934745 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.934350 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b54d755d-knnrx" event={"ID":"bda00eb1-73a1-4a21-ad07-08567cd1cbcc","Type":"ContainerStarted","Data":"934fd10971408a77f9d3ba38170e65f63d7cd5a66fef2b025df40d932368849f"} Jul 21 12:34:24.956200 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:24.956136 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b54d755d-knnrx" podStartSLOduration=0.956115943 podStartE2EDuration="956.115943ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:24.955577664 +0000 UTC m=+412.478305774" watchObservedRunningTime="2026-07-21 12:34:24.956115943 +0000 UTC m=+412.478843659" Jul 21 12:34:25.003446 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:25.003413 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:25.108064 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:25.108024 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:27.722307 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:27.722256 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:28.028666 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:28.028568 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:29.928380 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:29.928345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:30.180504 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:30.180402 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:31.614733 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.614697 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg"] Jul 21 12:34:31.617100 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.617079 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:31.619678 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.619650 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 12:34:31.619830 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.619788 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-v5ckd\"" Jul 21 12:34:31.620618 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.620599 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 12:34:31.620705 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.620624 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 12:34:31.637925 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.637891 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg"] Jul 21 12:34:31.672406 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.672355 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhxft\" (UniqueName: \"kubernetes.io/projected/414884d8-0651-4a90-8c99-23c77ef5fe79-kube-api-access-lhxft\") pod \"dns-operator-controller-manager-864dd78c4c-mt5vg\" (UID: \"414884d8-0651-4a90-8c99-23c77ef5fe79\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:31.773670 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.773631 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhxft\" (UniqueName: \"kubernetes.io/projected/414884d8-0651-4a90-8c99-23c77ef5fe79-kube-api-access-lhxft\") pod \"dns-operator-controller-manager-864dd78c4c-mt5vg\" (UID: \"414884d8-0651-4a90-8c99-23c77ef5fe79\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:31.783691 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.783654 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhxft\" (UniqueName: \"kubernetes.io/projected/414884d8-0651-4a90-8c99-23c77ef5fe79-kube-api-access-lhxft\") pod \"dns-operator-controller-manager-864dd78c4c-mt5vg\" (UID: \"414884d8-0651-4a90-8c99-23c77ef5fe79\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:31.928053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:31.928022 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:32.073031 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:32.072983 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg"] Jul 21 12:34:32.075594 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:34:32.075557 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414884d8_0651_4a90_8c99_23c77ef5fe79.slice/crio-5bfd7e3e6154b9570fb7492290d4400bf1c1ec7c3b347f55ce331a084f3e38e9 WatchSource:0}: Error finding container 5bfd7e3e6154b9570fb7492290d4400bf1c1ec7c3b347f55ce331a084f3e38e9: Status 404 returned error can't find the container with id 5bfd7e3e6154b9570fb7492290d4400bf1c1ec7c3b347f55ce331a084f3e38e9 Jul 21 12:34:32.923879 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:32.923833 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:32.980394 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:32.980359 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" event={"ID":"414884d8-0651-4a90-8c99-23c77ef5fe79","Type":"ContainerStarted","Data":"5bfd7e3e6154b9570fb7492290d4400bf1c1ec7c3b347f55ce331a084f3e38e9"} Jul 21 12:34:33.021863 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:33.021818 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:33.347424 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:33.347374 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:33.439950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:33.439909 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:34.548166 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.548127 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:34.548662 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.548199 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:34.553673 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.553644 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:34.664187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.664158 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 12:34:34.992472 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.992430 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" event={"ID":"414884d8-0651-4a90-8c99-23c77ef5fe79","Type":"ContainerStarted","Data":"8eb244de8acdba43ed9b9e6648678ec652d0b76df246f6ae4b7ba09502e64920"} Jul 21 12:34:34.996529 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:34.996500 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b54d755d-knnrx" Jul 21 12:34:35.010931 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.010848 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" podStartSLOduration=1.427112516 podStartE2EDuration="4.010830669s" totalImagesPullingTime="2.583718153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:35.009959212 +0000 UTC m=+422.532686942" watchObservedRunningTime="2026-07-21 12:34:35.010830669 +0000 UTC m=+422.533558385" Jul 21 12:34:35.057460 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.057424 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:34:35.220678 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.220639 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:35.321202 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.321099 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:35.422032 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.421987 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:35.522797 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.522759 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:35.899787 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:35.899754 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:36.000127 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:36.000090 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:38.369800 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:38.369747 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:38.540652 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:38.540619 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:41.616438 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:41.616400 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:41.718192 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:41.718151 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:45.459323 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:45.459284 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:45.591787 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:45.591743 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:46.168462 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.168420 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc"] Jul 21 12:34:46.171343 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.171313 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.174937 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.174909 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d4b6n\"" Jul 21 12:34:46.196568 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.196532 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc"] Jul 21 12:34:46.284005 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.283969 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc"] Jul 21 12:34:46.284327 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:46.284303 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-b4257], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" podUID="dfb087d6-0ee8-4500-9225-37e8cbdbdf30" Jul 21 12:34:46.306204 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.306163 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4257\" (UniqueName: \"kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.306409 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.306208 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.376923 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.376888 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc"] Jul 21 12:34:46.390440 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.390409 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc"] Jul 21 12:34:46.407680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.407643 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4257\" (UniqueName: \"kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.407680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.407685 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.408122 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.408104 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.411813 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:46.411789 2584 projected.go:196] Error preparing data for projected volume kube-api-access-b4257 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-140-37.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object Jul 21 12:34:46.411991 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:46.411860 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257 podName:dfb087d6-0ee8-4500-9225-37e8cbdbdf30 nodeName:}" failed. No retries permitted until 2026-07-21 12:34:46.911841655 +0000 UTC m=+434.434569350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b4257" (UniqueName: "kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257") pod "kuadrant-operator-controller-manager-55467d9f7-ggpbc" (UID: "dfb087d6-0ee8-4500-9225-37e8cbdbdf30") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-140-37.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object Jul 21 12:34:46.426766 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.426728 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:34:46.430185 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.430157 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.442097 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.442062 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:34:46.508475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.508440 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.508867 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.508557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjlm\" (UniqueName: \"kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.609165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.609120 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjlm\" (UniqueName: \"kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.609405 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.609259 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.609676 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.609655 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.621443 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.621408 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjlm\" (UniqueName: \"kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-b6sln\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.743680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.743578 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:46.906868 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.906839 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:34:46.909146 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:34:46.909115 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c57b74_3ac2_4c6b_a74d_bfb51f34edc5.slice/crio-a6b081d4feb4d068e1f913779fb5df0e18a58be4b2217b178c9f951bbd60e106 WatchSource:0}: Error finding container a6b081d4feb4d068e1f913779fb5df0e18a58be4b2217b178c9f951bbd60e106: Status 404 returned error can't find the container with id a6b081d4feb4d068e1f913779fb5df0e18a58be4b2217b178c9f951bbd60e106 Jul 21 12:34:46.911952 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:46.911925 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4257\" (UniqueName: \"kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257\") pod \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:46.920407 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:46.920382 2584 projected.go:196] Error preparing data for projected volume kube-api-access-b4257 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-ggpbc" not found Jul 21 12:34:46.920538 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:46.920450 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257 podName:dfb087d6-0ee8-4500-9225-37e8cbdbdf30 nodeName:}" failed. No retries permitted until 2026-07-21 12:34:47.920433641 +0000 UTC m=+435.443161346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4257" (UniqueName: "kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257") pod "kuadrant-operator-controller-manager-55467d9f7-ggpbc" (UID: "dfb087d6-0ee8-4500-9225-37e8cbdbdf30") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-ggpbc" not found Jul 21 12:34:47.005873 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.005788 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mt5vg" Jul 21 12:34:47.051255 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.051192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" event={"ID":"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5","Type":"ContainerStarted","Data":"a6b081d4feb4d068e1f913779fb5df0e18a58be4b2217b178c9f951bbd60e106"} Jul 21 12:34:47.051255 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.051247 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:47.059051 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.059021 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:47.062266 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:47.062236 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="dfb087d6-0ee8-4500-9225-37e8cbdbdf30" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:47.113562 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.113525 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume\") pod \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\" (UID: \"dfb087d6-0ee8-4500-9225-37e8cbdbdf30\") " Jul 21 12:34:47.113843 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.113821 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfb087d6-0ee8-4500-9225-37e8cbdbdf30" (UID: "dfb087d6-0ee8-4500-9225-37e8cbdbdf30"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:34:47.113931 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.113918 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4257\" (UniqueName: \"kubernetes.io/projected/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-kube-api-access-b4257\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:34:47.113968 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.113934 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfb087d6-0ee8-4500-9225-37e8cbdbdf30-extensions-socket-volume\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:34:47.877546 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.877515 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:47.973257 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:47.973200 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:48.056043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:48.056010 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:48.058391 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:48.058363 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="dfb087d6-0ee8-4500-9225-37e8cbdbdf30" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:48.064116 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:34:48.064082 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-ggpbc\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="dfb087d6-0ee8-4500-9225-37e8cbdbdf30" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ggpbc" Jul 21 12:34:48.966590 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:48.966556 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfb087d6-0ee8-4500-9225-37e8cbdbdf30" path="/var/lib/kubelet/pods/dfb087d6-0ee8-4500-9225-37e8cbdbdf30/volumes" Jul 21 12:34:53.083520 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.083482 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" event={"ID":"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5","Type":"ContainerStarted","Data":"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4"} Jul 21 12:34:53.083937 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.083725 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:34:53.104649 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.104590 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" podStartSLOduration=1.952145073 podStartE2EDuration="7.104566321s" totalImagesPullingTime="5.152421248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:53.103731185 +0000 UTC m=+440.626458895" watchObservedRunningTime="2026-07-21 12:34:53.104566321 +0000 UTC m=+440.627294033" Jul 21 12:34:53.338276 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.338180 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:53.396028 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.395992 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:53.752329 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.752289 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:53.853401 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:53.853367 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:57.759442 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:57.759346 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:34:57.837611 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:34:57.837574 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:00.082948 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.082910 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7976b7bd5-bv2zt" podUID="fb86c6ed-eacd-45d2-884d-80a0be574d63" containerName="console" containerID="cri-o://e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f" gracePeriod=15 Jul 21 12:35:00.334966 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.334895 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:35:00.442608 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442568 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.442823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442632 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.442823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442706 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.442823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442731 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.442823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442760 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.442823 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442806 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r46tk\" (UniqueName: \"kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.443077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.442860 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert\") pod \"fb86c6ed-eacd-45d2-884d-80a0be574d63\" (UID: \"fb86c6ed-eacd-45d2-884d-80a0be574d63\") " Jul 21 12:35:00.443077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443045 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:35:00.443179 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443121 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config" (OuterVolumeSpecName: "console-config") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:35:00.443179 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443169 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:35:00.443397 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443375 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.443397 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443393 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-service-ca\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.443397 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443402 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-trusted-ca-bundle\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.443589 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.443410 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:35:00.445045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.445019 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:35:00.445165 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.445146 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:35:00.445252 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.445186 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk" (OuterVolumeSpecName: "kube-api-access-r46tk") pod "fb86c6ed-eacd-45d2-884d-80a0be574d63" (UID: "fb86c6ed-eacd-45d2-884d-80a0be574d63"). InnerVolumeSpecName "kube-api-access-r46tk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:35:00.544187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.544143 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.544187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.544183 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb86c6ed-eacd-45d2-884d-80a0be574d63-console-oauth-config\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.544187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.544192 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb86c6ed-eacd-45d2-884d-80a0be574d63-oauth-serving-cert\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:00.544187 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:00.544200 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r46tk\" (UniqueName: \"kubernetes.io/projected/fb86c6ed-eacd-45d2-884d-80a0be574d63-kube-api-access-r46tk\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:01.123411 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.123372 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb86c6ed-eacd-45d2-884d-80a0be574d63" containerID="e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f" exitCode=0 Jul 21 12:35:01.123411 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.123415 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7976b7bd5-bv2zt" Jul 21 12:35:01.123866 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.123458 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7976b7bd5-bv2zt" event={"ID":"fb86c6ed-eacd-45d2-884d-80a0be574d63","Type":"ContainerDied","Data":"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f"} Jul 21 12:35:01.123866 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.123515 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7976b7bd5-bv2zt" event={"ID":"fb86c6ed-eacd-45d2-884d-80a0be574d63","Type":"ContainerDied","Data":"9755f67577b518353595e8621d914d3b6e490036646382c177fe77d47f559f80"} Jul 21 12:35:01.123866 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.123543 2584 scope.go:117] "RemoveContainer" containerID="e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f" Jul 21 12:35:01.132569 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.132547 2584 scope.go:117] "RemoveContainer" containerID="e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f" Jul 21 12:35:01.132892 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:35:01.132873 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f\": container with ID starting with e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f not found: ID does not exist" containerID="e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f" Jul 21 12:35:01.132955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.132902 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f"} err="failed to get container status \"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f\": rpc error: code = NotFound desc = could not find container \"e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f\": container with ID starting with e87454e6eeb0d1f263b9b8cb3099bb00f47c68cafee8265abdb71b2ef0b1848f not found: ID does not exist" Jul 21 12:35:01.143796 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.143766 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:35:01.148262 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:01.148238 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7976b7bd5-bv2zt"] Jul 21 12:35:02.965574 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:02.965539 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb86c6ed-eacd-45d2-884d-80a0be574d63" path="/var/lib/kubelet/pods/fb86c6ed-eacd-45d2-884d-80a0be574d63/volumes" Jul 21 12:35:04.091235 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:04.091176 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:35:05.757178 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:05.757141 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:05.857345 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:05.857311 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:07.748117 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:07.748069 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:07.803917 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:07.803860 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:09.647731 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.647691 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:35:09.648339 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.647938 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" podUID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" containerName="manager" containerID="cri-o://f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4" gracePeriod=10 Jul 21 12:35:09.906876 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.906807 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:35:09.943643 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.943596 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume\") pod \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " Jul 21 12:35:09.943825 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.943693 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqjlm\" (UniqueName: \"kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm\") pod \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\" (UID: \"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5\") " Jul 21 12:35:09.944168 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.944145 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" (UID: "70c57b74-3ac2-4c6b-a74d-bfb51f34edc5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:35:09.946073 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:09.946039 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm" (OuterVolumeSpecName: "kube-api-access-cqjlm") pod "70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" (UID: "70c57b74-3ac2-4c6b-a74d-bfb51f34edc5"). InnerVolumeSpecName "kube-api-access-cqjlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:35:10.044469 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.044432 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqjlm\" (UniqueName: \"kubernetes.io/projected/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-kube-api-access-cqjlm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:10.044469 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.044463 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5-extensions-socket-volume\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:35:10.169348 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.169247 2584 generic.go:356] "Generic (PLEG): container finished" podID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" containerID="f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4" exitCode=0 Jul 21 12:35:10.169348 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.169299 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" Jul 21 12:35:10.169348 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.169336 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" event={"ID":"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5","Type":"ContainerDied","Data":"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4"} Jul 21 12:35:10.169642 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.169399 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln" event={"ID":"70c57b74-3ac2-4c6b-a74d-bfb51f34edc5","Type":"ContainerDied","Data":"a6b081d4feb4d068e1f913779fb5df0e18a58be4b2217b178c9f951bbd60e106"} Jul 21 12:35:10.169642 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.169428 2584 scope.go:117] "RemoveContainer" containerID="f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4" Jul 21 12:35:10.178584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.178566 2584 scope.go:117] "RemoveContainer" containerID="f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4" Jul 21 12:35:10.178883 ip-10-0-140-37 kubenswrapper[2584]: E0721 12:35:10.178866 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4\": container with ID starting with f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4 not found: ID does not exist" containerID="f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4" Jul 21 12:35:10.178944 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.178896 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4"} err="failed to get container status \"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4\": rpc error: code = NotFound desc = could not find container \"f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4\": container with ID starting with f87809dc664c1207885a5b6a10863474a3797bbee61ac2d08f9c561717c6d8d4 not found: ID does not exist" Jul 21 12:35:10.195423 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.195384 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:35:10.199986 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.199947 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-b6sln"] Jul 21 12:35:10.966164 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:10.966129 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" path="/var/lib/kubelet/pods/70c57b74-3ac2-4c6b-a74d-bfb51f34edc5/volumes" Jul 21 12:35:26.070242 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070190 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg"] Jul 21 12:35:26.070679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070587 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb86c6ed-eacd-45d2-884d-80a0be574d63" containerName="console" Jul 21 12:35:26.070679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070599 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb86c6ed-eacd-45d2-884d-80a0be574d63" containerName="console" Jul 21 12:35:26.070679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070609 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" containerName="manager" Jul 21 12:35:26.070679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070615 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" containerName="manager" Jul 21 12:35:26.070679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070673 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c57b74-3ac2-4c6b-a74d-bfb51f34edc5" containerName="manager" Jul 21 12:35:26.070836 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.070685 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb86c6ed-eacd-45d2-884d-80a0be574d63" containerName="console" Jul 21 12:35:26.073820 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.073800 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.076714 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.076681 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 12:35:26.076908 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.076868 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x4dlb\"" Jul 21 12:35:26.090082 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.090049 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg"] Jul 21 12:35:26.106285 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.106242 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:26.193519 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193477 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193538 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dswx\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-kube-api-access-9dswx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193560 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193610 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193634 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193656 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193679 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193992 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193722 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/777c3364-4f85-42dd-a32d-0a102990cc5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.193992 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.193853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295009 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.294963 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295046 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dswx\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-kube-api-access-9dswx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295077 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295125 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295174 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295566 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295293 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295566 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295329 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/777c3364-4f85-42dd-a32d-0a102990cc5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295566 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295391 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295566 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295506 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295755 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295576 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295755 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295676 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.295755 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.295721 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.296105 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.296078 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/777c3364-4f85-42dd-a32d-0a102990cc5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.297697 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.297671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.297809 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.297774 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.303934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.303904 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.304077 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.304024 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dswx\" (UniqueName: \"kubernetes.io/projected/777c3364-4f85-42dd-a32d-0a102990cc5f-kube-api-access-9dswx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hx6tg\" (UID: \"777c3364-4f85-42dd-a32d-0a102990cc5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.384782 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.384678 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:26.533620 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:26.533589 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg"] Jul 21 12:35:26.535829 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:35:26.535797 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777c3364_4f85_42dd_a32d_0a102990cc5f.slice/crio-60236940f5f3f2cc89498599de50abd261e0f0cde26165297393c9a061d2ed2c WatchSource:0}: Error finding container 60236940f5f3f2cc89498599de50abd261e0f0cde26165297393c9a061d2ed2c: Status 404 returned error can't find the container with id 60236940f5f3f2cc89498599de50abd261e0f0cde26165297393c9a061d2ed2c Jul 21 12:35:27.259571 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:27.259530 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" event={"ID":"777c3364-4f85-42dd-a32d-0a102990cc5f","Type":"ContainerStarted","Data":"60236940f5f3f2cc89498599de50abd261e0f0cde26165297393c9a061d2ed2c"} Jul 21 12:35:27.296741 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:27.296703 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:30.181099 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.181056 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:35:30.181412 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.181137 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:35:30.181412 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.181165 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:35:30.278086 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.278045 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" event={"ID":"777c3364-4f85-42dd-a32d-0a102990cc5f","Type":"ContainerStarted","Data":"f95fa6af86ebae212a2b635244bdb658f1664b1515cb895f31afa7db760b6e4d"} Jul 21 12:35:30.298724 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.298648 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" podStartSLOduration=0.655948062 podStartE2EDuration="4.298624423s" totalImagesPullingTime="3.642676361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:35:30.298142966 +0000 UTC m=+477.820870682" watchObservedRunningTime="2026-07-21 12:35:30.298624423 +0000 UTC m=+477.821352140" Jul 21 12:35:30.384905 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:30.384845 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:31.330535 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.330497 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:31.390750 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.390717 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:31.449243 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.449182 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:31.523692 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.523661 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:31.638704 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.638613 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:35:31.642426 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.642401 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.644997 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.644970 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 12:35:31.645146 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.645081 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ckjqb\"" Jul 21 12:35:31.650835 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.650800 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:35:31.688917 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.688883 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:35:31.750396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.750351 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqv64\" (UniqueName: \"kubernetes.io/projected/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-kube-api-access-dqv64\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.750396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.750403 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-config-file\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.851340 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.851290 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-config-file\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.851571 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.851433 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqv64\" (UniqueName: \"kubernetes.io/projected/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-kube-api-access-dqv64\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.852156 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.852128 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-config-file\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.861174 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.861134 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqv64\" (UniqueName: \"kubernetes.io/projected/a32ed0ce-9e45-4c93-ae4d-435e3be85d49-kube-api-access-dqv64\") pod \"limitador-limitador-54d6d988d5-r9sw8\" (UID: \"a32ed0ce-9e45-4c93-ae4d-435e3be85d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:31.954797 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:31.954758 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:32.097960 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.097890 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:35:32.100852 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:35:32.100823 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32ed0ce_9e45_4c93_ae4d_435e3be85d49.slice/crio-dec9197c4b824a2dd3a52730214696a812b1aeff71b588b52b7bf4613ef99519 WatchSource:0}: Error finding container dec9197c4b824a2dd3a52730214696a812b1aeff71b588b52b7bf4613ef99519: Status 404 returned error can't find the container with id dec9197c4b824a2dd3a52730214696a812b1aeff71b588b52b7bf4613ef99519 Jul 21 12:35:32.192319 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.192278 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:32.293199 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.293098 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:32.296301 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.296264 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" event={"ID":"a32ed0ce-9e45-4c93-ae4d-435e3be85d49","Type":"ContainerStarted","Data":"dec9197c4b824a2dd3a52730214696a812b1aeff71b588b52b7bf4613ef99519"} Jul 21 12:35:32.296534 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.296517 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:32.297717 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.297698 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:35:32.379295 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.379257 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:35:32.383834 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.383808 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:35:32.386396 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.386371 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rhvw\"" Jul 21 12:35:32.389689 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.389664 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:35:32.457043 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.457003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzfr\" (UniqueName: \"kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr\") pod \"authorino-75d556bd77-7d6ln\" (UID: \"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd\") " pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:35:32.558385 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.558123 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzfr\" (UniqueName: \"kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr\") pod \"authorino-75d556bd77-7d6ln\" (UID: \"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd\") " pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:35:32.566889 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.566857 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzfr\" (UniqueName: \"kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr\") pod \"authorino-75d556bd77-7d6ln\" (UID: \"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd\") " pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:35:32.694973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.694924 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:35:32.877200 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.877134 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:35:32.879122 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:32.878979 2584 scope.go:117] "RemoveContainer" containerID="24707052377ff18221e74d69ff71404c0cb80ed6f5ff358c23d6e20da04e9575" Jul 21 12:35:33.304304 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:33.304258 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7d6ln" event={"ID":"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd","Type":"ContainerStarted","Data":"d61f7478f49708a44819aad5a06b1acfa2c72599ff7881e18dff084009fa6d7a"} Jul 21 12:35:37.325727 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:37.325681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" event={"ID":"a32ed0ce-9e45-4c93-ae4d-435e3be85d49","Type":"ContainerStarted","Data":"e91e175a6576e924d43ee53e235a9bfebf7bdfa71c55e1da8bd34c308102552b"} Jul 21 12:35:37.326267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:37.325841 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:37.327264 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:37.327241 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7d6ln" event={"ID":"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd","Type":"ContainerStarted","Data":"2bf98db7ff6110a3f9723f6aba535b875a43b0e8eaf08502a8959d6fe4da1c9f"} Jul 21 12:35:37.343088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:37.343013 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" podStartSLOduration=1.701184805 podStartE2EDuration="6.342991765s" totalImagesPullingTime="4.64180696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:35:37.342067116 +0000 UTC m=+484.864794833" watchObservedRunningTime="2026-07-21 12:35:37.342991765 +0000 UTC m=+484.865719527" Jul 21 12:35:37.357920 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:37.357861 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7d6ln" podStartSLOduration=1.492011588 podStartE2EDuration="5.357845101s" totalImagesPullingTime="3.865833513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:35:37.356127435 +0000 UTC m=+484.878855152" watchObservedRunningTime="2026-07-21 12:35:37.357845101 +0000 UTC m=+484.880572818" Jul 21 12:35:48.332987 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:48.332957 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-r9sw8" Jul 21 12:35:52.651547 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:52.651501 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:52.751526 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:52.751483 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:53.160450 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:53.160412 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:35:53.558130 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:35:53.558095 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:04.721629 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:04.721586 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:05.350795 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:05.350752 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:06.947130 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:06.947093 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:36:06.947581 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:06.947316 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7d6ln" podUID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" containerName="authorino" containerID="cri-o://2bf98db7ff6110a3f9723f6aba535b875a43b0e8eaf08502a8959d6fe4da1c9f" gracePeriod=30 Jul 21 12:36:07.472040 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.471987 2584 generic.go:356] "Generic (PLEG): container finished" podID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" containerID="2bf98db7ff6110a3f9723f6aba535b875a43b0e8eaf08502a8959d6fe4da1c9f" exitCode=0 Jul 21 12:36:07.472256 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.472063 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7d6ln" event={"ID":"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd","Type":"ContainerDied","Data":"2bf98db7ff6110a3f9723f6aba535b875a43b0e8eaf08502a8959d6fe4da1c9f"} Jul 21 12:36:07.725681 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.725581 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:36:07.796811 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.796774 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btzfr\" (UniqueName: \"kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr\") pod \"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd\" (UID: \"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd\") " Jul 21 12:36:07.798945 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.798909 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr" (OuterVolumeSpecName: "kube-api-access-btzfr") pod "22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" (UID: "22862619-a3cd-4ef5-bd3f-b3bf2fb479bd"). InnerVolumeSpecName "kube-api-access-btzfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:36:07.897684 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:07.897643 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btzfr\" (UniqueName: \"kubernetes.io/projected/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd-kube-api-access-btzfr\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 12:36:08.479584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.479543 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7d6ln" event={"ID":"22862619-a3cd-4ef5-bd3f-b3bf2fb479bd","Type":"ContainerDied","Data":"d61f7478f49708a44819aad5a06b1acfa2c72599ff7881e18dff084009fa6d7a"} Jul 21 12:36:08.479584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.479585 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7d6ln" Jul 21 12:36:08.480111 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.479608 2584 scope.go:117] "RemoveContainer" containerID="2bf98db7ff6110a3f9723f6aba535b875a43b0e8eaf08502a8959d6fe4da1c9f" Jul 21 12:36:08.500926 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.500880 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:36:08.504256 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.504225 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7d6ln"] Jul 21 12:36:08.965360 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:08.965324 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" path="/var/lib/kubelet/pods/22862619-a3cd-4ef5-bd3f-b3bf2fb479bd/volumes" Jul 21 12:36:11.864341 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:11.864300 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:11.964015 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:11.963981 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:12.037428 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:12.037394 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:12.307269 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:12.307201 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:15.170763 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:15.170722 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:15.272789 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:15.272753 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:27.079691 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:27.079594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:46.352462 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:46.352419 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:46.497411 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:46.497367 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:46.938324 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:46.938280 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:47.022311 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:47.022273 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:47.442307 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:47.442274 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:47.543835 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:47.543795 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:47.682944 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:47.682905 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:36:51.915504 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:51.915463 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:54.862119 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:54.862076 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:55.711080 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.711035 2584 generic.go:356] "Generic (PLEG): container finished" podID="777c3364-4f85-42dd-a32d-0a102990cc5f" containerID="f95fa6af86ebae212a2b635244bdb658f1664b1515cb895f31afa7db760b6e4d" exitCode=0 Jul 21 12:36:55.711308 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.711106 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" event={"ID":"777c3364-4f85-42dd-a32d-0a102990cc5f","Type":"ContainerDied","Data":"f95fa6af86ebae212a2b635244bdb658f1664b1515cb895f31afa7db760b6e4d"} Jul 21 12:36:55.711609 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.711578 2584 scope.go:117] "RemoveContainer" containerID="f95fa6af86ebae212a2b635244bdb658f1664b1515cb895f31afa7db760b6e4d" Jul 21 12:36:55.712707 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.712677 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:36:55.712764 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.712751 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:36:55.712804 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:55.712788 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 12:36:56.718469 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:56.718437 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" event={"ID":"777c3364-4f85-42dd-a32d-0a102990cc5f","Type":"ContainerStarted","Data":"4f2bb2a583250579048deaf95d8bbd2aa7320ef67adc2f1542ea360d31a3d330"} Jul 21 12:36:56.752071 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:56.752033 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:56.847590 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:56.847558 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:36:57.385755 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:57.385712 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:36:57.385978 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:57.385947 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hx6tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 21 12:36:57.386039 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:57.386012 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" podUID="777c3364-4f85-42dd-a32d-0a102990cc5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 21 12:36:58.385950 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:58.385907 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hx6tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 21 12:36:58.386507 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:58.385988 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" podUID="777c3364-4f85-42dd-a32d-0a102990cc5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 21 12:36:59.385137 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:59.385097 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hx6tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 21 12:36:59.385368 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:36:59.385161 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" podUID="777c3364-4f85-42dd-a32d-0a102990cc5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 21 12:37:00.386088 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:00.386042 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hx6tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 21 12:37:00.386634 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:00.386113 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" podUID="777c3364-4f85-42dd-a32d-0a102990cc5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 21 12:37:01.389136 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:01.389101 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:37:01.389598 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:01.389407 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:37:01.390282 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:01.390263 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hx6tg" Jul 21 12:37:32.869963 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:32.869932 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:37:32.876557 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:32.876527 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:37:40.129306 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:40.129263 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:37:40.691745 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:40.691706 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:37:43.108680 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:43.108643 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:37:43.790976 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:43.790939 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:37:51.537146 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:51.537102 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:37:52.690457 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:52.690413 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:37:58.166492 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:58.166402 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:37:58.270899 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:37:58.270860 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:38:14.310429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:14.310378 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:38:16.093518 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:16.093481 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:38:29.584900 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.584864 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp"] Jul 21 12:38:29.585429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.585289 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" containerName="authorino" Jul 21 12:38:29.585429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.585303 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" containerName="authorino" Jul 21 12:38:29.585429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.585370 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="22862619-a3cd-4ef5-bd3f-b3bf2fb479bd" containerName="authorino" Jul 21 12:38:29.587730 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.587708 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.590272 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.590248 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 12:38:29.590412 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.590249 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 12:38:29.591227 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.591197 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7sbsq\"" Jul 21 12:38:29.591488 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.591472 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 12:38:29.598031 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.598004 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp"] Jul 21 12:38:29.621177 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.621144 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:38:29.717538 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.717485 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:38:29.748882 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.748841 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.749094 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.748891 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.749094 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.748950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.749094 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.749002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39cfc508-1e12-4aaa-b572-239800b97a72-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.749094 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.749027 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.749359 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.749164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n8hc\" (UniqueName: \"kubernetes.io/projected/39cfc508-1e12-4aaa-b572-239800b97a72-kube-api-access-8n8hc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850142 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850031 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n8hc\" (UniqueName: \"kubernetes.io/projected/39cfc508-1e12-4aaa-b572-239800b97a72-kube-api-access-8n8hc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850142 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850112 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850142 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850142 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850168 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850189 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39cfc508-1e12-4aaa-b572-239800b97a72-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.850449 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850259 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.858267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.850556 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.858267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.851119 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.858267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.851297 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.858267 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.853528 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39cfc508-1e12-4aaa-b572-239800b97a72-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.858591 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.858539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n8hc\" (UniqueName: \"kubernetes.io/projected/39cfc508-1e12-4aaa-b572-239800b97a72-kube-api-access-8n8hc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.859327 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.859305 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39cfc508-1e12-4aaa-b572-239800b97a72-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cbbqp\" (UID: \"39cfc508-1e12-4aaa-b572-239800b97a72\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:29.898785 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:29.898743 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:30.044737 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:30.044708 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp"] Jul 21 12:38:30.047073 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:38:30.047047 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cfc508_1e12_4aaa_b572_239800b97a72.slice/crio-ef0558545e170502a96a73d23966ba29609f08967d7939d034785357c9b14fdf WatchSource:0}: Error finding container ef0558545e170502a96a73d23966ba29609f08967d7939d034785357c9b14fdf: Status 404 returned error can't find the container with id ef0558545e170502a96a73d23966ba29609f08967d7939d034785357c9b14fdf Jul 21 12:38:30.048431 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:30.048411 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:38:30.173090 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:30.173001 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" event={"ID":"39cfc508-1e12-4aaa-b572-239800b97a72","Type":"ContainerStarted","Data":"ef0558545e170502a96a73d23966ba29609f08967d7939d034785357c9b14fdf"} Jul 21 12:38:32.184571 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:32.184529 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" event={"ID":"39cfc508-1e12-4aaa-b572-239800b97a72","Type":"ContainerStarted","Data":"864407eeb2e88a4b9ed1b9b45fd3abb5b8537540057245a34c145c89fd57476c"} Jul 21 12:38:32.196392 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:32.196361 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:38:32.207708 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:32.207650 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" podStartSLOduration=1.817303668 podStartE2EDuration="3.207633275s" totalImagesPullingTime="1.390329607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:38:32.205829571 +0000 UTC m=+659.728557288" watchObservedRunningTime="2026-07-21 12:38:32.207633275 +0000 UTC m=+659.730360991" Jul 21 12:38:38.877739 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.877699 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r"] Jul 21 12:38:38.880706 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.880681 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.883413 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.883388 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 12:38:38.891734 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.891709 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r"] Jul 21 12:38:38.907545 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.907511 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:38:38.935953 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.935919 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlpq\" (UniqueName: \"kubernetes.io/projected/8e005f76-128e-4b59-b225-85e5af5e9d1a-kube-api-access-2nlpq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.936117 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.935961 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.936117 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.935996 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.936117 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.936042 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e005f76-128e-4b59-b225-85e5af5e9d1a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.936117 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.936082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:38.936271 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:38.936122 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.036817 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.036782 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.036846 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e005f76-128e-4b59-b225-85e5af5e9d1a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.036891 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.036938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037045 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.037033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlpq\" (UniqueName: \"kubernetes.io/projected/8e005f76-128e-4b59-b225-85e5af5e9d1a-kube-api-access-2nlpq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037285 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.037061 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037346 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.037291 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037442 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.037416 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.037684 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.037665 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.039380 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.039360 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e005f76-128e-4b59-b225-85e5af5e9d1a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.039509 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.039484 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e005f76-128e-4b59-b225-85e5af5e9d1a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.045733 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.045706 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlpq\" (UniqueName: \"kubernetes.io/projected/8e005f76-128e-4b59-b225-85e5af5e9d1a-kube-api-access-2nlpq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2947r\" (UID: \"8e005f76-128e-4b59-b225-85e5af5e9d1a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.191475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.191437 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:39.341979 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.341945 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r"] Jul 21 12:38:39.344003 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:38:39.343973 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e005f76_128e_4b59_b225_85e5af5e9d1a.slice/crio-9fd47324bb7ebb1df169c0058c39f1f892671611b98dbb67d3c8da7bc1754dc8 WatchSource:0}: Error finding container 9fd47324bb7ebb1df169c0058c39f1f892671611b98dbb67d3c8da7bc1754dc8: Status 404 returned error can't find the container with id 9fd47324bb7ebb1df169c0058c39f1f892671611b98dbb67d3c8da7bc1754dc8 Jul 21 12:38:39.899203 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.899160 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:39.899736 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.899308 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:39.911949 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:39.911916 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:40.223420 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:40.223382 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" event={"ID":"8e005f76-128e-4b59-b225-85e5af5e9d1a","Type":"ContainerStarted","Data":"9fd47324bb7ebb1df169c0058c39f1f892671611b98dbb67d3c8da7bc1754dc8"} Jul 21 12:38:40.234892 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:40.234857 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cbbqp" Jul 21 12:38:41.230304 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:41.230259 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" event={"ID":"8e005f76-128e-4b59-b225-85e5af5e9d1a","Type":"ContainerStarted","Data":"aeeba4ffa35c8a33728a96d64fc2263e06ca3fb63f6c95954762bfb9f539db12"} Jul 21 12:38:41.251275 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:41.251200 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" podStartSLOduration=2.112817256 podStartE2EDuration="3.251182471s" totalImagesPullingTime="1.138365215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:38:41.24970974 +0000 UTC m=+668.772437458" watchObservedRunningTime="2026-07-21 12:38:41.251182471 +0000 UTC m=+668.773910186" Jul 21 12:38:41.879372 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:41.879334 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:38:49.191720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:49.191674 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:49.191720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:49.191729 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:49.204362 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:49.204333 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:38:49.281351 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:38:49.281320 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2947r" Jul 21 12:39:05.774644 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:05.774605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:07.940074 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:07.940029 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:08.041800 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:08.041756 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:55.867203 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.867166 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8fb5fdbfc-58vlm"] Jul 21 12:39:55.870592 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.870574 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:55.873857 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.873835 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rhvw\"" Jul 21 12:39:55.873972 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.873857 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 12:39:55.878822 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.878800 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8fb5fdbfc-58vlm"] Jul 21 12:39:55.979675 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.979628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-tls-cert\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:55.979883 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:55.979682 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4qcf\" (UniqueName: \"kubernetes.io/projected/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-kube-api-access-w4qcf\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.080667 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.080626 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-tls-cert\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.080876 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.080677 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qcf\" (UniqueName: \"kubernetes.io/projected/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-kube-api-access-w4qcf\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.083105 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.083083 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-tls-cert\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.089658 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.089631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qcf\" (UniqueName: \"kubernetes.io/projected/32d68a95-8c52-4a0e-ba16-3a0bd334ea1c-kube-api-access-w4qcf\") pod \"authorino-8fb5fdbfc-58vlm\" (UID: \"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c\") " pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.180676 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.180643 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" Jul 21 12:39:56.316940 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.316908 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8fb5fdbfc-58vlm"] Jul 21 12:39:56.319265 ip-10-0-140-37 kubenswrapper[2584]: W0721 12:39:56.319233 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32d68a95_8c52_4a0e_ba16_3a0bd334ea1c.slice/crio-63816b24a709f49143376b926af09ad69f67f72dad934548962b1e66651e16b1 WatchSource:0}: Error finding container 63816b24a709f49143376b926af09ad69f67f72dad934548962b1e66651e16b1: Status 404 returned error can't find the container with id 63816b24a709f49143376b926af09ad69f67f72dad934548962b1e66651e16b1 Jul 21 12:39:56.596015 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:56.595916 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" event={"ID":"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c","Type":"ContainerStarted","Data":"63816b24a709f49143376b926af09ad69f67f72dad934548962b1e66651e16b1"} Jul 21 12:39:57.602296 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:57.602250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" event={"ID":"32d68a95-8c52-4a0e-ba16-3a0bd334ea1c","Type":"ContainerStarted","Data":"40350a2c598ed2a71f0b754e79b95d66a72c4b8fe75aa3ca8505fb016e9bbfc6"} Jul 21 12:39:57.625461 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:57.625392 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8fb5fdbfc-58vlm" podStartSLOduration=2.162378321 podStartE2EDuration="2.625368766s" totalImagesPullingTime="462.990445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:39:57.622364808 +0000 UTC m=+745.145092525" watchObservedRunningTime="2026-07-21 12:39:57.625368766 +0000 UTC m=+745.148096483" Jul 21 12:39:58.180809 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:58.180772 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:58.257788 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:58.257751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:59.404385 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:59.404350 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:39:59.500049 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:39:59.500013 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:28.182795 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:40:28.182744 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:40:37.797864 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:40:37.797828 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:40:47.393272 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:40:47.393234 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:41:23.508496 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:41:23.508457 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:05.512189 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:05.512152 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:06.047542 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:06.047506 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:27.889784 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:27.889694 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:42:32.909403 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:32.909375 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:42:32.913736 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:32.913708 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:42:40.288726 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:40.288684 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:42:44.779003 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:44.778965 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:44.877757 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:44.877722 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:45.014652 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:45.014605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:51.028061 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:51.028023 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:51.135233 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:51.135189 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:51.229451 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:51.229415 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:51.406593 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:51.406494 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:57.104084 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:57.104043 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:58.482994 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:42:58.482952 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:14.383442 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:43:14.383400 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:43:23.884588 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:43:23.884547 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:43:34.182681 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:43:34.182643 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:43:42.680655 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:43:42.680614 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:43:52.593266 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:43:52.593196 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:44:04.083275 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:04.083176 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:44:14.778889 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:14.778847 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:44:24.020517 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:24.020481 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:44:26.202588 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:26.202553 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:42.890473 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:42.890427 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:42.991465 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:42.991429 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:47.135321 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:47.135275 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:44:50.384712 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:50.384668 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:50.443940 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:50.443903 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:59.383302 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:44:59.383255 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:45:24.608994 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:45:24.608903 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:45:34.384594 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:45:34.384554 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:45:48.987702 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:45:48.987650 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:45:57.996841 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:45:57.996791 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:46:07.290693 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:46:07.290648 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:46:18.684543 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:46:18.684500 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:46:28.088819 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:46:28.088776 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:47:32.952798 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:47:32.952768 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:47:32.953350 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:47:32.952841 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:48:16.090475 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:48:16.090438 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:48:27.994327 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:48:27.994242 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:48:36.605984 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:48:36.605935 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:48:49.190382 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:48:49.190344 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:00.486101 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:00.486060 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:11.097065 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:11.097025 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:19.782978 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:19.782939 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:30.683354 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:30.683314 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:30.807155 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:30.807123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:42.385916 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:42.385876 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:49:51.279907 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:49:51.279870 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:50:04.598140 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:04.598046 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:50:23.293059 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:23.293022 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:50:32.988993 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:32.988954 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:50:39.229684 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:39.229650 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:43.685260 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:43.685205 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:50:53.025390 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:53.025345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:59.188714 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:50:59.188672 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:51:12.284044 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:12.284009 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:13.441233 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:13.441177 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:15.988584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:15.988543 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:51:28.692505 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:28.692414 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:51:35.578115 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:35.578076 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:51:51.189709 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:51:51.189665 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:03.184416 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:03.184375 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:11.977118 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:11.977078 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:14.149309 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:14.149274 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:52:21.992113 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:21.992076 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:30.895765 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:30.895727 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:32.983162 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:32.983133 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:52:32.985358 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:32.985335 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:52:39.684436 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:39.684393 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:52:53.579904 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:52:53.579863 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:02.085688 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:02.085592 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:09.138335 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:09.138297 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:12.790584 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:12.790549 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:23.285095 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:23.285043 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:35.387010 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:35.386967 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:45.392125 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:45.392087 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:53:57.479601 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:53:57.479561 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:54:07.504813 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:07.504772 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:54:18.888724 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:18.888688 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:54:24.634951 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:24.634877 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:24.725343 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:24.725312 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:25.124325 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:25.124290 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:30.899082 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:30.899035 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:30.996050 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:30.996017 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:31.132479 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:31.132421 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:31.580651 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:31.580616 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:31.741147 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:31.741114 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:34.214541 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:34.214504 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:35.986865 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:35.986829 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:41.087953 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:54:41.087914 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:55:10.470958 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:10.470925 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:13.285701 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:13.285660 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:55:14.998690 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:14.998661 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:15.165757 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:15.165714 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:15.386661 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:15.386580 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:15.636429 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:15.636393 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:15.729245 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:15.729196 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:15.857555 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:15.857524 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:16.232679 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:16.232630 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:16.633493 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:16.633404 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:18.616874 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:18.616834 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:18.769913 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:18.769883 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:18.879774 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:18.879690 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:19.289647 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:19.289617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:19.388401 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:19.388371 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:24.482784 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:24.482744 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:55:54.221183 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:54.221155 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:57.293825 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:57.293790 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:55:58.787016 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:58.786983 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:58.936603 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:58.936570 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:59.208586 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:59.208545 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:59.402555 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:59.402527 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:59.866074 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:55:59.866033 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:00.527776 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:00.527741 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:02.703415 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:02.703381 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:02.915508 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:02.915472 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:03.026493 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:03.026402 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:03.817334 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:03.817301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:03.947474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:03.947433 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:04.061854 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:04.061819 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:08.527437 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:08.527401 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:56:39.149519 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:39.149481 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:42.187881 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:42.187837 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:56:43.648370 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:43.648332 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:44.946991 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:44.946956 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:45.172516 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:45.172482 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:45.428883 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:45.428853 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:45.787108 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:45.787001 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:45.977370 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:45.977335 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:47.852999 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:47.852964 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:47.959840 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:47.959790 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:48.048831 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:48.048790 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:48.219833 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:48.219800 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:48.502973 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:48.502844 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:48.608720 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:48.608687 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:53.876793 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:56:53.876759 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:57:23.160740 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:23.160694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:26.188560 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:26.188469 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:57:27.490008 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:27.489977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:27.592053 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:27.592006 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:27.843700 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:27.843607 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:27.978278 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:27.978240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:28.228526 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:28.228490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:28.765564 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:28.765530 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:28.901653 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:28.901625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:29.016019 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:29.015926 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:30.510664 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:30.510627 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:33.014002 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:33.013972 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:57:33.016620 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:33.016592 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 12:57:37.707743 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:37.707707 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:37.840980 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:37.840945 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:38.058550 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:38.058462 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:38.215472 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:38.215436 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:38.612802 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:38.612771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:49.506332 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:49.506289 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:57:49.606230 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:57:49.606175 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:04.543898 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:04.543858 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:11.276504 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:11.276468 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:11.381446 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:11.381411 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:11.477049 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:11.477011 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:27.364170 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:27.364138 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:33.513030 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:33.512994 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:33.620231 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:33.620175 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:33.714448 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:33.714418 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:44.334579 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:44.334545 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:45.072317 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:45.072272 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:52.172474 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:52.172434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:52.276039 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:52.276006 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:52.375299 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:52.375263 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:52.474935 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:52.474894 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:53.320768 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:53.320732 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:53.420277 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:53.420242 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:53.519530 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:53.519496 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:55.218894 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:55.218801 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:58:56.193961 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:56.193921 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:58:56.388903 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:58:56.388874 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:02.653736 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:02.653699 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:02.752586 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:02.752545 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:03.356037 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:03.356008 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:06.583922 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:06.583880 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:59:09.091392 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:09.091358 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:09.191173 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:09.191142 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:09.339813 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:09.339774 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:09.677874 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:09.677839 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:10.489375 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:10.489338 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:13.253417 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:13.253388 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:13.351620 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:13.351593 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:13.451161 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:13.451127 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:13.751183 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:13.751144 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:13.908375 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:13.908340 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:14.010930 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:14.010835 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:17.287934 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:17.287894 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:59:17.657465 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:17.657365 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:18.395866 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:18.395821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:18.548615 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:18.548575 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:18.849955 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:18.849908 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:24.649501 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:24.649469 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:24.719637 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:24.719610 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:35.557501 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:35.557459 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:38.378635 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:38.378591 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 12:59:40.126358 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:40.126319 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:41.679756 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:41.679708 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.208769 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.208728 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.309397 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.309362 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.508992 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.508896 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.740049 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.740018 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.882004 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.881912 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:46.981867 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:46.981831 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:47.119483 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:47.119450 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:52.829412 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:52.829379 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:52.975532 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:52.975497 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:53.071758 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:53.071721 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:53.203188 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:53.203160 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:59.389610 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:59.389579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:59.545507 ip-10-0-140-37 kubenswrapper[2584]: I0721 12:59:59.545472 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:06.442102 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:06.442069 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:06.768191 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:06.768102 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:06.944130 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:06.944094 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:09.338688 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:09.338656 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:09.421982 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:09.421949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:09.527978 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:09.527942 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:09.906270 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:09.906184 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:10.007539 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:10.007505 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:10.220096 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:10.220064 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:13.864966 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:13.864931 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:16.185145 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:16.185100 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:00:41.024627 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:41.024546 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:00:44.090895 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:44.090850 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:00:59.186875 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:00:59.186835 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:01:11.511381 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:11.511346 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:01:17.085970 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:17.085927 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:01:18.423574 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:18.423541 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:01:23.036265 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:23.036194 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:01:25.266843 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:25.266801 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:01:27.681729 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:27.681688 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:01:30.348671 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:30.348632 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:01:48.672645 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:01:48.672609 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:06.399132 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:06.399053 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:07.818240 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:07.818176 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:13.846702 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:13.846671 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:13.921405 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:13.921345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:14.152783 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:14.152698 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:14.383410 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:14.383379 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:15.444267 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:15.444235 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:15.541252 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:15.541202 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:15.641208 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:15.641171 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:16.391591 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:16.391560 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:16.486609 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:16.486579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:16.586993 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:16.586954 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:17.933051 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:17.933015 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:02:18.048549 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:18.048517 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:23.925504 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:23.925467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:23.987494 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:23.987457 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:24.715700 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:24.715673 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:27.292336 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:27.292298 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:02:28.389102 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:28.389069 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:28.626254 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:28.626205 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:30.613612 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:30.613574 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:30.720050 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:30.720007 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:30.840809 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:30.840769 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:33.047887 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:33.047862 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:02:33.052834 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:33.052806 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:02:41.640055 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:41.640016 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:41.741379 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:41.741351 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:51.974388 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:51.974352 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:52.053073 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:02:52.053026 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:02.092770 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:02.092732 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:02.734207 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:02.734170 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:09.525869 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:09.525831 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:09.635683 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:09.635644 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:09.911435 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:09.911349 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:10.345768 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:10.345736 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:10.791198 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:10.791164 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:13.224679 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:13.224642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:13.342885 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:13.342855 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:13.485374 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:13.485287 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:14.054893 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:14.054856 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:14.333611 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:14.333528 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:17.776562 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:17.776532 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:18.135116 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:18.135023 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:18.457158 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:18.457124 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:18.487964 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:18.487927 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:03:18.824174 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:18.824094 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:21.270703 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:21.270665 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:21.562363 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:21.562271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:21.819541 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:21.819443 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:21.962644 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:21.962611 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:22.209920 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:22.209888 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:22.329053 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:22.329022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:25.753918 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:25.753831 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:34.591373 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.591325 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:03:34.594716 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.594688 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:03:34.594848 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.594820 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.597646 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.597620 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-52a2ba\"/\"kube-root-ca.crt\"" Jul 21 13:03:34.597646 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.597633 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-52a2ba\"/\"test-model-52a2ba-kserve-self-signed-certs\"" Jul 21 13:03:34.597868 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.597620 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-52a2ba\"/\"default-dockercfg-hnzq8\"" Jul 21 13:03:34.597868 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.597664 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-52a2ba\"/\"openshift-service-ca.crt\"" Jul 21 13:03:34.604770 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.604736 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:03:34.621146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621027 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.621146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621075 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.621146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.621549 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.621549 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621315 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.621549 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.621362 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xk9q\" (UniqueName: \"kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.694170 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.694136 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:34.721964 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.721914 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.721964 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.721961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722315 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722002 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722315 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722047 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722315 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722315 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xk9q\" (UniqueName: \"kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722563 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722409 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722563 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.722563 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.722535 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.724733 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.724706 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.724919 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.724899 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.731937 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.731888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xk9q\" (UniqueName: \"kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q\") pod \"test-model-52a2ba-kserve-6c768b995f-qfjxj\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.907057 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.906943 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:34.940274 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:34.940202 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:35.040939 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:35.040911 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:35.064624 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:35.064540 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:03:35.067892 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:03:35.067843 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded919b37_c8cb_4b1c_873a_0c208886eff8.slice/crio-d9c10e61f080b60ccba20d753ef5f51f82c56eca72707d4626d127adbf55e3f3 WatchSource:0}: Error finding container d9c10e61f080b60ccba20d753ef5f51f82c56eca72707d4626d127adbf55e3f3: Status 404 returned error can't find the container with id d9c10e61f080b60ccba20d753ef5f51f82c56eca72707d4626d127adbf55e3f3 Jul 21 13:03:35.523070 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:35.523034 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" event={"ID":"ed919b37-c8cb-4b1c-873a-0c208886eff8","Type":"ContainerStarted","Data":"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4"} Jul 21 13:03:35.523070 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:35.523080 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" event={"ID":"ed919b37-c8cb-4b1c-873a-0c208886eff8","Type":"ContainerStarted","Data":"d9c10e61f080b60ccba20d753ef5f51f82c56eca72707d4626d127adbf55e3f3"} Jul 21 13:03:42.976812 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:42.976778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:43.077674 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:43.077642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:44.907279 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:44.907242 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:44.907680 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:44.907307 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:44.920091 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:44.920060 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:44.937953 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:44.937893 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" podStartSLOduration=10.937877414999999 podStartE2EDuration="10.937877415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:03:35.539183934 +0000 UTC m=+2163.061911651" watchObservedRunningTime="2026-07-21 13:03:44.937877415 +0000 UTC m=+2172.460605161" Jul 21 13:03:45.586092 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:45.586053 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:03:47.988276 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:03:47.988239 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:04:05.126963 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:05.126925 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:10.088141 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:10.088070 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:04:23.885865 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:23.885814 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:04:45.482383 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:45.482341 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:04:45.507701 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:45.507672 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:47.801499 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:47.801467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:49.891987 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:49.891940 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:04:55.324899 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:55.324798 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:57.109373 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:04:57.109335 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:00.503994 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.503948 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" podUID="ed919b37-c8cb-4b1c-873a-0c208886eff8" containerName="main" containerID="cri-o://99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4" gracePeriod=45 Jul 21 13:05:00.766553 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.766471 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:05:00.880987 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.880947 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881006 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881041 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xk9q\" (UniqueName: \"kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881090 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881136 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881152 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm\") pod \"ed919b37-c8cb-4b1c-873a-0c208886eff8\" (UID: \"ed919b37-c8cb-4b1c-873a-0c208886eff8\") " Jul 21 13:05:00.881496 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881305 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:05:00.881496 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881450 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:05:00.881496 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881473 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home" (OuterVolumeSpecName: "home") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:05:00.881614 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881539 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-model-cache\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.881614 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881556 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-home\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.881614 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.881566 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-tmp-dir\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.883531 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.883508 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:05:00.883531 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.883522 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q" (OuterVolumeSpecName: "kube-api-access-7xk9q") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "kube-api-access-7xk9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:05:00.883667 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.883569 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm" (OuterVolumeSpecName: "dshm") pod "ed919b37-c8cb-4b1c-873a-0c208886eff8" (UID: "ed919b37-c8cb-4b1c-873a-0c208886eff8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:05:00.969648 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.969605 2584 generic.go:356] "Generic (PLEG): container finished" podID="ed919b37-c8cb-4b1c-873a-0c208886eff8" containerID="99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4" exitCode=0 Jul 21 13:05:00.969874 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.969738 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" Jul 21 13:05:00.972598 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.972560 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" event={"ID":"ed919b37-c8cb-4b1c-873a-0c208886eff8","Type":"ContainerDied","Data":"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4"} Jul 21 13:05:00.972598 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.972604 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj" event={"ID":"ed919b37-c8cb-4b1c-873a-0c208886eff8","Type":"ContainerDied","Data":"d9c10e61f080b60ccba20d753ef5f51f82c56eca72707d4626d127adbf55e3f3"} Jul 21 13:05:00.972854 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.972622 2584 scope.go:117] "RemoveContainer" containerID="99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4" Jul 21 13:05:00.982178 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.982147 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed919b37-c8cb-4b1c-873a-0c208886eff8-tls-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.982339 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.982199 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xk9q\" (UniqueName: \"kubernetes.io/projected/ed919b37-c8cb-4b1c-873a-0c208886eff8-kube-api-access-7xk9q\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.982339 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.982223 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed919b37-c8cb-4b1c-873a-0c208886eff8-dshm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:05:00.983590 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.983571 2584 scope.go:117] "RemoveContainer" containerID="99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4" Jul 21 13:05:00.983896 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:05:00.983875 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4\": container with ID starting with 99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4 not found: ID does not exist" containerID="99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4" Jul 21 13:05:00.983954 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.983906 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4"} err="failed to get container status \"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4\": rpc error: code = NotFound desc = could not find container \"99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4\": container with ID starting with 99b371ebd592adf6c2accc0642c3d5540ab28e3f279db286456af95b1b565ce4 not found: ID does not exist" Jul 21 13:05:00.998106 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:00.998069 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:05:01.002480 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:01.002445 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-52a2ba/test-model-52a2ba-kserve-6c768b995f-qfjxj"] Jul 21 13:05:01.712069 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:01.712031 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:02.135737 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:02.135581 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:02.784645 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:02.784605 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:05:02.965453 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:02.965417 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed919b37-c8cb-4b1c-873a-0c208886eff8" path="/var/lib/kubelet/pods/ed919b37-c8cb-4b1c-873a-0c208886eff8/volumes" Jul 21 13:05:07.795955 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:07.795913 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:07.894497 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:07.894457 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:08.101754 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:08.101647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:08.410440 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:08.410331 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:09.318974 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:09.318937 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:09.413756 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:09.413719 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:19.007484 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:19.007449 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:22.095603 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:22.095562 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:05:23.578664 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:23.578613 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:23.998058 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:23.997964 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:30.140458 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:30.140417 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:30.205859 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:30.205817 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:30.390745 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:30.390635 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:30.661464 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:30.661368 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:31.029623 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:31.029587 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:31.354604 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:31.354503 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:31.744924 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:31.744882 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:33.941317 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:33.941275 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:34.021560 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:34.021522 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:34.093880 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:34.093840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:34.277141 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:34.277041 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:34.744005 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:34.743960 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:34.843957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:34.843891 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:38.478324 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:38.478287 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:39.020310 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:39.020264 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:39.374199 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:39.374112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:39.555550 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:39.555512 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:41.583514 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:41.583477 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:05:42.003769 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:42.003730 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:42.085157 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:42.085103 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:42.168262 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:42.168222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:42.659845 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:42.659771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:42.756953 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:42.756907 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:46.555577 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:46.555542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:55.381882 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:55.381840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:55.481544 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:55.481507 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:05:57.986700 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:05:57.986659 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:06:03.451943 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:03.451905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:03.552414 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:03.552379 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:05.442700 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:05.442625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:05.544650 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:05.544613 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:10.694566 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:10.694529 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:14.385772 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:14.385728 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:06:14.876944 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:14.876907 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:15.277754 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:15.277697 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:21.218447 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:21.218401 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:21.635082 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:21.634963 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:21.736247 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:21.736176 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:21.836071 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:21.836029 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:22.384105 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:22.384069 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:22.485508 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:22.485468 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:37.381963 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:37.381858 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:41.929311 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:41.929275 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:42.085246 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:42.085194 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:06:42.308679 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:42.308559 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:48.212621 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:48.212578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:48.313341 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:48.313301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:48.468495 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:48.468397 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:48.706982 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:48.706944 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:49.055905 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:49.055870 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:49.387112 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:49.387020 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:49.837969 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:49.837932 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:52.287351 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:52.287311 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:52.363148 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:52.363104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:52.437534 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:52.437484 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:52.874731 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:52.874701 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:52.976610 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:52.976577 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:53.076454 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:53.076412 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:56.791147 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:56.791116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:57.129445 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:57.129350 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:57.430722 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:57.430688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:06:57.513640 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:06:57.513599 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:00.184948 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:00.184905 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:07:00.400789 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:00.400751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:00.517879 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:00.517778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:00.604276 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:00.604246 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:00.732517 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:00.732466 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:01.227103 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:01.227070 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:01.324348 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:01.324307 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:04.838631 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:04.838601 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:13.928327 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:13.928288 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:14.030558 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:14.030521 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:14.671406 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:14.671361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:14.780550 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:14.780511 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:07:23.168059 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:23.168024 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:23.270334 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:23.270301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:29.204964 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:29.204930 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:29.310493 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:29.310460 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:33.079971 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:33.079929 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:07:33.085306 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:33.085265 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:07:34.787796 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.787750 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:07:34.788351 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.788332 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed919b37-c8cb-4b1c-873a-0c208886eff8" containerName="main" Jul 21 13:07:34.788351 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.788352 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed919b37-c8cb-4b1c-873a-0c208886eff8" containerName="main" Jul 21 13:07:34.788439 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.788417 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed919b37-c8cb-4b1c-873a-0c208886eff8" containerName="main" Jul 21 13:07:34.791458 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.791438 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.808094 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.808057 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"default-dockercfg-24thh\"" Jul 21 13:07:34.808718 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.808691 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"auth-test-model-3cf59c-kserve-self-signed-certs\"" Jul 21 13:07:34.809895 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.809871 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"openshift-service-ca.crt\"" Jul 21 13:07:34.810416 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.810397 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"kube-root-ca.crt\"" Jul 21 13:07:34.814935 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.814904 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:07:34.827673 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.827638 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:34.883720 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883676 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.883957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883743 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.883957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883766 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.883957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883791 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.883957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883850 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.884149 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.883956 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwwq\" (UniqueName: \"kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985383 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985343 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985600 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985396 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985600 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwwq\" (UniqueName: \"kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985600 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985503 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985600 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985574 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985600 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985599 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985890 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.985943 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.985882 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.986077 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.986057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.987870 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.987849 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.988047 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.988029 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:34.994411 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:34.994383 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwwq\" (UniqueName: \"kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq\") pod \"auth-test-model-3cf59c-kserve-75659dbdc-j9ktk\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:35.122507 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:35.122399 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:35.272069 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:35.272036 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:07:35.274697 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:07:35.274659 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod467228a5_f9b8_483b_abd9_86e3f62dc564.slice/crio-b115e6d00557462443837b9a6f97e173b466f39e388cdb71cb47606a184a8f36 WatchSource:0}: Error finding container b115e6d00557462443837b9a6f97e173b466f39e388cdb71cb47606a184a8f36: Status 404 returned error can't find the container with id b115e6d00557462443837b9a6f97e173b466f39e388cdb71cb47606a184a8f36 Jul 21 13:07:35.757836 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:35.757791 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" event={"ID":"467228a5-f9b8-483b-abd9-86e3f62dc564","Type":"ContainerStarted","Data":"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc"} Jul 21 13:07:35.758037 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:35.757850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" event={"ID":"467228a5-f9b8-483b-abd9-86e3f62dc564","Type":"ContainerStarted","Data":"b115e6d00557462443837b9a6f97e173b466f39e388cdb71cb47606a184a8f36"} Jul 21 13:07:35.775630 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:35.775567 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" podStartSLOduration=1.775547524 podStartE2EDuration="1.775547524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:07:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:07:35.773545957 +0000 UTC m=+2403.296273799" watchObservedRunningTime="2026-07-21 13:07:35.775547524 +0000 UTC m=+2403.298275300" Jul 21 13:07:38.192857 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:38.192815 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:07:45.123476 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:45.123434 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:45.123476 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:45.123484 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:45.136208 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:45.136166 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:07:45.821459 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:07:45.821421 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:08:01.541737 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:08:01.541642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:05.088047 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:08:05.087994 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:08:22.504687 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:08:22.504637 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:08:41.977441 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:08:41.977397 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:08:55.494181 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:08:55.494137 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:09:12.287832 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:09:12.287785 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:09:15.529521 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:09:15.529480 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:15.595226 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:09:15.595173 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:30.989328 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:09:30.989230 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:09:49.889807 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:09:49.889769 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:10:09.688616 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:09.688578 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:10:14.171281 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:14.171244 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:10:14.188987 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:14.188944 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:16.193193 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:16.193151 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:27.681594 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:27.681552 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:10:29.191729 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.191686 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" podUID="467228a5-f9b8-483b-abd9-86e3f62dc564" containerName="main" containerID="cri-o://1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc" gracePeriod=45 Jul 21 13:10:29.455969 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.455879 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:10:29.564371 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564330 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564371 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564379 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564609 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564413 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564609 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564609 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564523 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdwwq\" (UniqueName: \"kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564609 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564574 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home\") pod \"467228a5-f9b8-483b-abd9-86e3f62dc564\" (UID: \"467228a5-f9b8-483b-abd9-86e3f62dc564\") " Jul 21 13:10:29.564823 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564681 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:10:29.564823 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564787 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache" (OuterVolumeSpecName: "model-cache") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:10:29.564937 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564907 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home" (OuterVolumeSpecName: "home") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:10:29.564998 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.564986 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-model-cache\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.565035 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.565003 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-home\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.565035 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.565012 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-tmp-dir\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.566759 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.566722 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm" (OuterVolumeSpecName: "dshm") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:10:29.566887 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.566773 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:10:29.566887 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.566791 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq" (OuterVolumeSpecName: "kube-api-access-kdwwq") pod "467228a5-f9b8-483b-abd9-86e3f62dc564" (UID: "467228a5-f9b8-483b-abd9-86e3f62dc564"). InnerVolumeSpecName "kube-api-access-kdwwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:10:29.661146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.661109 2584 generic.go:356] "Generic (PLEG): container finished" podID="467228a5-f9b8-483b-abd9-86e3f62dc564" containerID="1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc" exitCode=0 Jul 21 13:10:29.661146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.661148 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" Jul 21 13:10:29.661386 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.661190 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" event={"ID":"467228a5-f9b8-483b-abd9-86e3f62dc564","Type":"ContainerDied","Data":"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc"} Jul 21 13:10:29.661386 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.661264 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk" event={"ID":"467228a5-f9b8-483b-abd9-86e3f62dc564","Type":"ContainerDied","Data":"b115e6d00557462443837b9a6f97e173b466f39e388cdb71cb47606a184a8f36"} Jul 21 13:10:29.661386 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.661287 2584 scope.go:117] "RemoveContainer" containerID="1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc" Jul 21 13:10:29.665545 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.665518 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdwwq\" (UniqueName: \"kubernetes.io/projected/467228a5-f9b8-483b-abd9-86e3f62dc564-kube-api-access-kdwwq\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.665545 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.665546 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467228a5-f9b8-483b-abd9-86e3f62dc564-tls-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.665705 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.665558 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467228a5-f9b8-483b-abd9-86e3f62dc564-dshm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:10:29.670674 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.670653 2584 scope.go:117] "RemoveContainer" containerID="1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc" Jul 21 13:10:29.671051 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:10:29.671027 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc\": container with ID starting with 1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc not found: ID does not exist" containerID="1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc" Jul 21 13:10:29.671108 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.671066 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc"} err="failed to get container status \"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc\": rpc error: code = NotFound desc = could not find container \"1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc\": container with ID starting with 1f8baf0df5b068669d233160f34d460baf02817cda8753cbf448e5c315bf26dc not found: ID does not exist" Jul 21 13:10:29.683792 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.683756 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:10:29.689398 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:29.689364 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/auth-test-model-3cf59c-kserve-75659dbdc-j9ktk"] Jul 21 13:10:30.965099 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:30.965059 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467228a5-f9b8-483b-abd9-86e3f62dc564" path="/var/lib/kubelet/pods/467228a5-f9b8-483b-abd9-86e3f62dc564/volumes" Jul 21 13:10:32.294688 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.294655 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:10:32.295089 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.295044 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467228a5-f9b8-483b-abd9-86e3f62dc564" containerName="main" Jul 21 13:10:32.295089 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.295056 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="467228a5-f9b8-483b-abd9-86e3f62dc564" containerName="main" Jul 21 13:10:32.295175 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.295143 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="467228a5-f9b8-483b-abd9-86e3f62dc564" containerName="main" Jul 21 13:10:32.299777 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.299750 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.302343 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.302319 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"openshift-service-ca.crt\"" Jul 21 13:10:32.303247 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.303208 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"kube-root-ca.crt\"" Jul 21 13:10:32.303398 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.303224 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"default-dockercfg-24thh\"" Jul 21 13:10:32.303475 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.303402 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"sub-test-model-3cf59c-kserve-self-signed-certs\"" Jul 21 13:10:32.307756 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.307726 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:10:32.327192 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.327157 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:32.391759 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.391696 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.391759 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.391775 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.392002 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.391845 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.392002 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.391947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.392002 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.391988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwnv5\" (UniqueName: \"kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.392113 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.392026 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.492978 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.492915 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.492978 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.492983 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwnv5\" (UniqueName: \"kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493302 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493022 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493302 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493051 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493302 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493302 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493577 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493416 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493577 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493519 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.493577 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.493549 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.495419 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.495389 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.495684 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.495664 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.502561 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.502535 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwnv5\" (UniqueName: \"kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5\") pod \"sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.611893 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.611772 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:32.758178 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:32.758151 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:10:32.760934 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:10:32.760905 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3861b0d2_858b_4d4c_b39a_145dbe09f5ee.slice/crio-ab6bd6fa23abf43d9ca8427da2702f752f79acaba7c97692c0c67bb13be3238d WatchSource:0}: Error finding container ab6bd6fa23abf43d9ca8427da2702f752f79acaba7c97692c0c67bb13be3238d: Status 404 returned error can't find the container with id ab6bd6fa23abf43d9ca8427da2702f752f79acaba7c97692c0c67bb13be3238d Jul 21 13:10:33.684410 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:33.684370 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" event={"ID":"3861b0d2-858b-4d4c-b39a-145dbe09f5ee","Type":"ContainerStarted","Data":"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3"} Jul 21 13:10:33.684900 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:33.684422 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" event={"ID":"3861b0d2-858b-4d4c-b39a-145dbe09f5ee","Type":"ContainerStarted","Data":"ab6bd6fa23abf43d9ca8427da2702f752f79acaba7c97692c0c67bb13be3238d"} Jul 21 13:10:33.702822 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:33.702765 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" podStartSLOduration=1.70274713 podStartE2EDuration="1.70274713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:10:33.700645343 +0000 UTC m=+2581.223373071" watchObservedRunningTime="2026-07-21 13:10:33.70274713 +0000 UTC m=+2581.225474889" Jul 21 13:10:36.280791 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:36.280748 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:10:42.612146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:42.612083 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:42.612146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:42.612161 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:42.625053 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:42.625022 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:42.739472 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:42.739440 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:10:57.060794 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:10:57.060708 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:11.818864 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:11.818832 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:15.610259 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:15.610208 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:11:23.877309 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:23.877252 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:23.931146 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:23.931107 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:39.986717 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:39.986669 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:11:45.699457 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:45.699419 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:58.290849 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:11:58.290806 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:12:04.889559 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:04.889518 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:12:04.909809 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:04.909777 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:05.799000 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:05.798961 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:14.801495 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:14.801452 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:14.901337 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:14.901301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:15.577881 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:15.577844 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:12:19.910553 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:19.910513 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" podUID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" containerName="main" containerID="cri-o://8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3" gracePeriod=45 Jul 21 13:12:20.169429 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.169354 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:12:20.211763 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.211725 2584 generic.go:356] "Generic (PLEG): container finished" podID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" containerID="8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3" exitCode=0 Jul 21 13:12:20.211957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.211780 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" Jul 21 13:12:20.211957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.211798 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" event={"ID":"3861b0d2-858b-4d4c-b39a-145dbe09f5ee","Type":"ContainerDied","Data":"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3"} Jul 21 13:12:20.211957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.211841 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8" event={"ID":"3861b0d2-858b-4d4c-b39a-145dbe09f5ee","Type":"ContainerDied","Data":"ab6bd6fa23abf43d9ca8427da2702f752f79acaba7c97692c0c67bb13be3238d"} Jul 21 13:12:20.211957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.211866 2584 scope.go:117] "RemoveContainer" containerID="8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3" Jul 21 13:12:20.220883 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.220860 2584 scope.go:117] "RemoveContainer" containerID="8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3" Jul 21 13:12:20.221225 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:12:20.221184 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3\": container with ID starting with 8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3 not found: ID does not exist" containerID="8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3" Jul 21 13:12:20.221330 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.221252 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3"} err="failed to get container status \"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3\": rpc error: code = NotFound desc = could not find container \"8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3\": container with ID starting with 8571e075d549fab6bd99a98bba74beb588018b0f18687ba989daf0b07e2012a3 not found: ID does not exist" Jul 21 13:12:20.266383 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266341 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwnv5\" (UniqueName: \"kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266383 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266390 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266615 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266416 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266615 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266472 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266615 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266533 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266615 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266592 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm\") pod \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\" (UID: \"3861b0d2-858b-4d4c-b39a-145dbe09f5ee\") " Jul 21 13:12:20.266828 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266686 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home" (OuterVolumeSpecName: "home") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:12:20.266828 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266711 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:12:20.266933 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266846 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:12:20.267015 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.266997 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-home\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.267081 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.267021 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tmp-dir\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.267081 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.267035 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-model-cache\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.268675 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.268655 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm" (OuterVolumeSpecName: "dshm") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:12:20.268755 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.268712 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5" (OuterVolumeSpecName: "kube-api-access-qwnv5") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "kube-api-access-qwnv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:12:20.268798 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.268766 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3861b0d2-858b-4d4c-b39a-145dbe09f5ee" (UID: "3861b0d2-858b-4d4c-b39a-145dbe09f5ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:12:20.368021 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.367975 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-tls-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.368021 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.368011 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-dshm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.368021 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.368020 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwnv5\" (UniqueName: \"kubernetes.io/projected/3861b0d2-858b-4d4c-b39a-145dbe09f5ee-kube-api-access-qwnv5\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:12:20.534311 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.534267 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:12:20.537091 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.537065 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/sub-test-model-3cf59c-kserve-8558b9f9f7-xdmk8"] Jul 21 13:12:20.964695 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:20.964657 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" path="/var/lib/kubelet/pods/3861b0d2-858b-4d4c-b39a-145dbe09f5ee/volumes" Jul 21 13:12:23.921062 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:23.921023 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:24.020578 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:24.020530 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:30.083358 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:30.083324 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:30.179461 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:30.179417 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:33.111362 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:33.111330 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:12:33.120764 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:33.120737 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlm2m_e2b79acb-4d11-4a73-bfc6-58811cdb8850/ovn-acl-logging/0.log" Jul 21 13:12:40.496009 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.495962 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:12:40.496659 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.496635 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" containerName="main" Jul 21 13:12:40.496751 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.496665 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" containerName="main" Jul 21 13:12:40.496802 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.496771 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3861b0d2-858b-4d4c-b39a-145dbe09f5ee" containerName="main" Jul 21 13:12:40.500298 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.500272 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.503078 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.503044 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"default-dockercfg-24thh\"" Jul 21 13:12:40.504002 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.503979 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"openshift-service-ca.crt\"" Jul 21 13:12:40.504116 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.504032 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"rate-test-model-3cf59c-kserve-self-signed-certs\"" Jul 21 13:12:40.504116 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.504097 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3cf59c\"/\"kube-root-ca.crt\"" Jul 21 13:12:40.511045 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.511019 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:12:40.547167 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.547130 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:40.670795 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.670758 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbj2g\" (UniqueName: \"kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.671012 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.670811 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.671012 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.670913 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.671012 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.670973 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.671012 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.671005 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.671223 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.671065 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772569 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772466 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772569 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772511 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772569 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772543 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772866 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772594 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbj2g\" (UniqueName: \"kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772866 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772635 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.772866 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772695 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.773025 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772943 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.773025 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.772999 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.773104 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.773081 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.775046 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.775024 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.775261 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.775246 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.782710 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.782681 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbj2g\" (UniqueName: \"kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g\") pod \"rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.810530 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.810495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:40.957142 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:40.957111 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:12:40.959280 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:12:40.959243 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9f8f9e4_ad15_4a03_b3b1_20b87d7fabef.slice/crio-883a2be5b4b86150105f39e8d6a6417926f965069852cc7eda6ed7bb8cfd1da9 WatchSource:0}: Error finding container 883a2be5b4b86150105f39e8d6a6417926f965069852cc7eda6ed7bb8cfd1da9: Status 404 returned error can't find the container with id 883a2be5b4b86150105f39e8d6a6417926f965069852cc7eda6ed7bb8cfd1da9 Jul 21 13:12:41.318681 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:41.318636 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" event={"ID":"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef","Type":"ContainerStarted","Data":"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c"} Jul 21 13:12:41.318681 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:41.318685 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" event={"ID":"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef","Type":"ContainerStarted","Data":"883a2be5b4b86150105f39e8d6a6417926f965069852cc7eda6ed7bb8cfd1da9"} Jul 21 13:12:41.338555 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:41.338496 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" podStartSLOduration=1.338480243 podStartE2EDuration="1.338480243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:12:41.336797163 +0000 UTC m=+2708.859524896" watchObservedRunningTime="2026-07-21 13:12:41.338480243 +0000 UTC m=+2708.861207959" Jul 21 13:12:43.888563 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:43.888521 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:12:50.811417 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:50.811374 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:50.811417 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:50.811429 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:50.824328 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:50.824291 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:12:51.379115 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:12:51.379086 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:13:18.929539 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:13:18.929495 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:22.699685 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:13:22.699637 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:13:41.684766 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:13:41.684722 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:13:55.480797 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:13:55.480699 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:14:03.231116 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:03.231079 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:14:03.253646 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:03.253609 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:04.311245 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:04.311178 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:08.802793 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.802751 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725295907120"] Jul 21 13:14:08.806649 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.806630 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:08.809676 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.809648 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rjjv6\"" Jul 21 13:14:08.813353 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.813325 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725295907120"] Jul 21 13:14:08.843123 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.843080 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmtr4\" (UniqueName: \"kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4\") pod \"test-curl-7509-139725295907120\" (UID: \"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf\") " pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:08.943742 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.943698 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmtr4\" (UniqueName: \"kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4\") pod \"test-curl-7509-139725295907120\" (UID: \"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf\") " pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:08.952784 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:08.952757 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmtr4\" (UniqueName: \"kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4\") pod \"test-curl-7509-139725295907120\" (UID: \"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf\") " pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:09.118106 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:09.118004 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:09.265392 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:09.265356 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725295907120"] Jul 21 13:14:09.269425 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:14:09.269392 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fc2b2c2_29aa_4850_bc0e_e1d65dfa0fdf.slice/crio-0b7652561faedadce1bb11f22ae104d042b371d05127d83ecbc1746c0ca580c2 WatchSource:0}: Error finding container 0b7652561faedadce1bb11f22ae104d042b371d05127d83ecbc1746c0ca580c2: Status 404 returned error can't find the container with id 0b7652561faedadce1bb11f22ae104d042b371d05127d83ecbc1746c0ca580c2 Jul 21 13:14:09.270690 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:09.270666 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:14:09.760508 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:09.760468 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725295907120" event={"ID":"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf","Type":"ContainerStarted","Data":"0b7652561faedadce1bb11f22ae104d042b371d05127d83ecbc1746c0ca580c2"} Jul 21 13:14:10.767828 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:10.767796 2584 generic.go:356] "Generic (PLEG): container finished" podID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" containerID="a38c020fe85de6418592267f69fd0fef884ab1cf29416994ec103f2c0c525098" exitCode=0 Jul 21 13:14:10.768321 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:10.767881 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725295907120" event={"ID":"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf","Type":"ContainerDied","Data":"a38c020fe85de6418592267f69fd0fef884ab1cf29416994ec103f2c0c525098"} Jul 21 13:14:11.914036 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.914001 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:11.940399 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.940373 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725295907120_4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf/test-curl-7509-139725295907120/0.log" Jul 21 13:14:11.963367 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.963336 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725295907120"] Jul 21 13:14:11.969053 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.969025 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725295907120"] Jul 21 13:14:11.974293 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.974267 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmtr4\" (UniqueName: \"kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4\") pod \"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf\" (UID: \"4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf\") " Jul 21 13:14:11.976557 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:11.976529 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4" (OuterVolumeSpecName: "kube-api-access-pmtr4") pod "4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" (UID: "4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf"). InnerVolumeSpecName "kube-api-access-pmtr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:14:12.075703 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.075605 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmtr4\" (UniqueName: \"kubernetes.io/projected/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf-kube-api-access-pmtr4\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:12.129843 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.129799 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:12.130306 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.130286 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" containerName="test-curl-7509-139725295907120" Jul 21 13:14:12.130306 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.130305 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" containerName="test-curl-7509-139725295907120" Jul 21 13:14:12.130483 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.130363 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" containerName="test-curl-7509-139725295907120" Jul 21 13:14:12.133526 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.133495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:12.138645 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.138613 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:12.176867 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.176826 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg895\" (UniqueName: \"kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895\") pod \"test-curl-7509-139725296088240\" (UID: \"52e23732-cae1-4a39-8297-f59cf4a5cc74\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:12.278185 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.278138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg895\" (UniqueName: \"kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895\") pod \"test-curl-7509-139725296088240\" (UID: \"52e23732-cae1-4a39-8297-f59cf4a5cc74\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:12.287832 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.287796 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg895\" (UniqueName: \"kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895\") pod \"test-curl-7509-139725296088240\" (UID: \"52e23732-cae1-4a39-8297-f59cf4a5cc74\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:12.445717 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.445679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:12.583018 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.582992 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:12.585388 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:14:12.585360 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52e23732_cae1_4a39_8297_f59cf4a5cc74.slice/crio-473d79282862970130cb3f0dc73d172221852d68c652575245165af7bd4c3522 WatchSource:0}: Error finding container 473d79282862970130cb3f0dc73d172221852d68c652575245165af7bd4c3522: Status 404 returned error can't find the container with id 473d79282862970130cb3f0dc73d172221852d68c652575245165af7bd4c3522 Jul 21 13:14:12.778860 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.778753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296088240" event={"ID":"52e23732-cae1-4a39-8297-f59cf4a5cc74","Type":"ContainerStarted","Data":"473d79282862970130cb3f0dc73d172221852d68c652575245165af7bd4c3522"} Jul 21 13:14:12.779931 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.779913 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b7652561faedadce1bb11f22ae104d042b371d05127d83ecbc1746c0ca580c2" Jul 21 13:14:12.780030 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.779956 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:12.791363 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:14:12.791335 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139725295907120\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:12.963161 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:14:12.963123 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139725295907120\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" pod="opendatahub/test-curl-7509-139725295907120" Jul 21 13:14:12.965453 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:12.965426 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf" path="/var/lib/kubelet/pods/4fc2b2c2-29aa-4850-bc0e-e1d65dfa0fdf/volumes" Jul 21 13:14:13.786010 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:13.785972 2584 generic.go:356] "Generic (PLEG): container finished" podID="52e23732-cae1-4a39-8297-f59cf4a5cc74" containerID="eec02000dd8e85a2d70418c02e62e2f716cb61b2f7f3a08b60fb25ec9c5671e1" exitCode=0 Jul 21 13:14:13.786202 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:13.786016 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296088240" event={"ID":"52e23732-cae1-4a39-8297-f59cf4a5cc74","Type":"ContainerDied","Data":"eec02000dd8e85a2d70418c02e62e2f716cb61b2f7f3a08b60fb25ec9c5671e1"} Jul 21 13:14:14.932492 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:14.932468 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:14.962477 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:14.962450 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725296088240_52e23732-cae1-4a39-8297-f59cf4a5cc74/test-curl-7509-139725296088240/0.log" Jul 21 13:14:14.988800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:14.988764 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:14.996007 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:14.995976 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:15.004075 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.004046 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg895\" (UniqueName: \"kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895\") pod \"52e23732-cae1-4a39-8297-f59cf4a5cc74\" (UID: \"52e23732-cae1-4a39-8297-f59cf4a5cc74\") " Jul 21 13:14:15.006185 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.006162 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895" (OuterVolumeSpecName: "kube-api-access-cg895") pod "52e23732-cae1-4a39-8297-f59cf4a5cc74" (UID: "52e23732-cae1-4a39-8297-f59cf4a5cc74"). InnerVolumeSpecName "kube-api-access-cg895". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:14:15.104907 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.104813 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg895\" (UniqueName: \"kubernetes.io/projected/52e23732-cae1-4a39-8297-f59cf4a5cc74-kube-api-access-cg895\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:15.119336 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.119296 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:15.119746 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.119731 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e23732-cae1-4a39-8297-f59cf4a5cc74" containerName="test-curl-7509-139725296088240" Jul 21 13:14:15.119787 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.119748 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e23732-cae1-4a39-8297-f59cf4a5cc74" containerName="test-curl-7509-139725296088240" Jul 21 13:14:15.119828 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.119818 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e23732-cae1-4a39-8297-f59cf4a5cc74" containerName="test-curl-7509-139725296088240" Jul 21 13:14:15.122918 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.122900 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.129723 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.129693 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:15.205676 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.205638 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27p7\" (UniqueName: \"kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7\") pod \"test-curl-7509-139725296088240\" (UID: \"e9f04686-2b57-4d8a-9b9d-0e6f294a8179\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.306994 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.306957 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27p7\" (UniqueName: \"kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7\") pod \"test-curl-7509-139725296088240\" (UID: \"e9f04686-2b57-4d8a-9b9d-0e6f294a8179\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.316090 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.316055 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27p7\" (UniqueName: \"kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7\") pod \"test-curl-7509-139725296088240\" (UID: \"e9f04686-2b57-4d8a-9b9d-0e6f294a8179\") " pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.433766 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.433732 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.778729 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.778637 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:15.781032 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:14:15.781002 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9f04686_2b57_4d8a_9b9d_0e6f294a8179.slice/crio-6f63dd7454f2413505b2bb5e3acfecab236d9f0ff45a2ffeb8652bcd5b0f4e62 WatchSource:0}: Error finding container 6f63dd7454f2413505b2bb5e3acfecab236d9f0ff45a2ffeb8652bcd5b0f4e62: Status 404 returned error can't find the container with id 6f63dd7454f2413505b2bb5e3acfecab236d9f0ff45a2ffeb8652bcd5b0f4e62 Jul 21 13:14:15.797029 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.796994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296088240" event={"ID":"e9f04686-2b57-4d8a-9b9d-0e6f294a8179","Type":"ContainerStarted","Data":"6f63dd7454f2413505b2bb5e3acfecab236d9f0ff45a2ffeb8652bcd5b0f4e62"} Jul 21 13:14:15.798280 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.798261 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="473d79282862970130cb3f0dc73d172221852d68c652575245165af7bd4c3522" Jul 21 13:14:15.798454 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.798321 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:15.810557 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:15.810521 2584 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7509-139725296088240" oldPodUID="52e23732-cae1-4a39-8297-f59cf4a5cc74" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" Jul 21 13:14:16.336063 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:16.335964 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:14:16.804957 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:16.804922 2584 generic.go:356] "Generic (PLEG): container finished" podID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" containerID="8ae18b7ede5439ee46dbf59aa2318b200956de951d4d3599aefec88764c1c7a4" exitCode=0 Jul 21 13:14:16.805131 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:16.805008 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296088240" event={"ID":"e9f04686-2b57-4d8a-9b9d-0e6f294a8179","Type":"ContainerDied","Data":"8ae18b7ede5439ee46dbf59aa2318b200956de951d4d3599aefec88764c1c7a4"} Jul 21 13:14:16.964880 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:16.964848 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52e23732-cae1-4a39-8297-f59cf4a5cc74" path="/var/lib/kubelet/pods/52e23732-cae1-4a39-8297-f59cf4a5cc74/volumes" Jul 21 13:14:17.947998 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:17.947967 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:17.971569 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:17.971539 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725296088240_e9f04686-2b57-4d8a-9b9d-0e6f294a8179/test-curl-7509-139725296088240/0.log" Jul 21 13:14:17.998229 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:17.998172 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:18.002693 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.002661 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725296088240"] Jul 21 13:14:18.032091 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.032058 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g27p7\" (UniqueName: \"kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7\") pod \"e9f04686-2b57-4d8a-9b9d-0e6f294a8179\" (UID: \"e9f04686-2b57-4d8a-9b9d-0e6f294a8179\") " Jul 21 13:14:18.034311 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.034285 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7" (OuterVolumeSpecName: "kube-api-access-g27p7") pod "e9f04686-2b57-4d8a-9b9d-0e6f294a8179" (UID: "e9f04686-2b57-4d8a-9b9d-0e6f294a8179"). InnerVolumeSpecName "kube-api-access-g27p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:14:18.133188 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.133082 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g27p7\" (UniqueName: \"kubernetes.io/projected/e9f04686-2b57-4d8a-9b9d-0e6f294a8179-kube-api-access-g27p7\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.252481 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.252438 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" podUID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" containerName="main" containerID="cri-o://12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c" gracePeriod=45 Jul 21 13:14:18.504601 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.504567 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:14:18.537415 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537371 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537593 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537446 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537593 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537474 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537593 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537521 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbj2g\" (UniqueName: \"kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537593 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537572 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537818 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537617 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs\") pod \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\" (UID: \"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef\") " Jul 21 13:14:18.537818 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537662 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home" (OuterVolumeSpecName: "home") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:14:18.537818 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537754 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:14:18.537966 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537951 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-home\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.538027 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537970 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tmp-dir\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.538027 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.537967 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:14:18.539794 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.539762 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm" (OuterVolumeSpecName: "dshm") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:14:18.539961 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.539937 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g" (OuterVolumeSpecName: "kube-api-access-cbj2g") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "kube-api-access-cbj2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:14:18.540010 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.539966 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" (UID: "e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:14:18.638871 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.638826 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbj2g\" (UniqueName: \"kubernetes.io/projected/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-kube-api-access-cbj2g\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.638871 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.638863 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-model-cache\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.638871 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.638874 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-tls-certs\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.638871 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.638884 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef-dshm\") on node \"ip-10-0-140-37.ec2.internal\" DevicePath \"\"" Jul 21 13:14:18.816613 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.816576 2584 generic.go:356] "Generic (PLEG): container finished" podID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" containerID="12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c" exitCode=0 Jul 21 13:14:18.816800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.816624 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" Jul 21 13:14:18.816800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.816658 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" event={"ID":"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef","Type":"ContainerDied","Data":"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c"} Jul 21 13:14:18.816800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.816701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t" event={"ID":"e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef","Type":"ContainerDied","Data":"883a2be5b4b86150105f39e8d6a6417926f965069852cc7eda6ed7bb8cfd1da9"} Jul 21 13:14:18.816800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.816724 2584 scope.go:117] "RemoveContainer" containerID="12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c" Jul 21 13:14:18.818260 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.818058 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:18.818260 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.818076 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f63dd7454f2413505b2bb5e3acfecab236d9f0ff45a2ffeb8652bcd5b0f4e62" Jul 21 13:14:18.827816 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.827795 2584 scope.go:117] "RemoveContainer" containerID="12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c" Jul 21 13:14:18.828200 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:14:18.828177 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c\": container with ID starting with 12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c not found: ID does not exist" containerID="12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c" Jul 21 13:14:18.828300 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.828266 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c"} err="failed to get container status \"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c\": rpc error: code = NotFound desc = could not find container \"12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c\": container with ID starting with 12193b1a398d531affe0cd6798018d9fda6dcb7545db38cf40fbf48bee62b51c not found: ID does not exist" Jul 21 13:14:18.845797 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.845766 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:14:18.849957 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:14:18.849924 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139725296088240\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:18.850371 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.850351 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3cf59c/rate-test-model-3cf59c-kserve-5dd5677d9f-64m6t"] Jul 21 13:14:18.851812 ip-10-0-140-37 kubenswrapper[2584]: E0721 13:14:18.851790 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139725296088240\" is forbidden: User \"system:node:ip-10-0-140-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-37.ec2.internal' and this object" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" pod="opendatahub/test-curl-7509-139725296088240" Jul 21 13:14:18.964800 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.964768 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" path="/var/lib/kubelet/pods/e9f04686-2b57-4d8a-9b9d-0e6f294a8179/volumes" Jul 21 13:14:18.965184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:18.965171 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" path="/var/lib/kubelet/pods/e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef/volumes" Jul 21 13:14:24.580948 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:24.580909 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:24.681105 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:24.681070 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:32.575054 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:32.574974 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:34.444098 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:34.444068 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:37.289183 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:37.289138 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:14:39.104314 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:39.104285 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:39.466480 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:39.466436 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:45.461197 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:45.461162 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:45.563038 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:45.562998 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:45.743556 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:45.743469 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:46.033292 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:46.033181 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:46.803435 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:46.803401 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:46.906468 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:46.906417 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:56.608410 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:56.608376 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:59.390813 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:14:59.390769 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-r9sw8"] Jul 21 13:15:01.241153 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:01.241122 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:01.681026 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:01.680996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:07.562450 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:07.562413 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:07.627678 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:07.627650 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:07.883634 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:07.883549 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:08.091960 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:08.091930 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:08.514755 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:08.514725 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8fb5fdbfc-58vlm_32d68a95-8c52-4a0e-ba16-3a0bd334ea1c/authorino/0.log" Jul 21 13:15:14.932108 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:14.932079 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-hkrlt_4a6e2033-527e-4719-9ab8-e9576135c34e/copy-manifests/0.log" Jul 21 13:15:14.945665 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:14.945635 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-hkrlt_4a6e2033-527e-4719-9ab8-e9576135c34e/manager/0.log" Jul 21 13:15:17.497605 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:17.497573 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8fb5fdbfc-58vlm_32d68a95-8c52-4a0e-ba16-3a0bd334ea1c/authorino/0.log" Jul 21 13:15:17.760748 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:17.760672 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mt5vg_414884d8-0651-4a90-8c99-23c77ef5fe79/manager/0.log" Jul 21 13:15:18.272209 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:18.272180 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r9sw8_a32ed0ce-9e45-4c93-ae4d-435e3be85d49/limitador/0.log" Jul 21 13:15:19.015380 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:19.015352 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-hz7t2_fd3ef682-d12a-4ad9-a603-159f2969bd01/kube-auth-proxy/0.log" Jul 21 13:15:19.268833 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:19.268748 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/1.log" Jul 21 13:15:19.633387 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:19.633310 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c4984944-x74sv_0d642c82-dbe3-43f3-b083-9cb442147639/router/0.log" Jul 21 13:15:20.490644 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:20.490613 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cbbqp_39cfc508-1e12-4aaa-b572-239800b97a72/main/0.log" Jul 21 13:15:20.608765 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:20.608738 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2947r_8e005f76-128e-4b59-b225-85e5af5e9d1a/main/0.log" Jul 21 13:15:26.706198 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:26.706113 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:29.832419 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:29.832386 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:29.839068 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:29.839040 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xzw56_0cf4f59d-684e-4266-a7f4-e9c233c4b1ff/global-pull-secret-syncer/0.log" Jul 21 13:15:29.895696 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:29.895662 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6k9r9_10a8417d-5d86-4639-ba93-7de9d9bb9395/konnectivity-agent/0.log" Jul 21 13:15:30.025122 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:30.025094 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-37.ec2.internal_a870da13b5e904749212ecd217225c3b/haproxy/0.log" Jul 21 13:15:34.821089 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:34.821054 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8fb5fdbfc-58vlm_32d68a95-8c52-4a0e-ba16-3a0bd334ea1c/authorino/0.log" Jul 21 13:15:34.963244 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:34.962070 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mt5vg_414884d8-0651-4a90-8c99-23c77ef5fe79/manager/0.log" Jul 21 13:15:35.307870 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:35.307840 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-r9sw8_a32ed0ce-9e45-4c93-ae4d-435e3be85d49/limitador/0.log" Jul 21 13:15:37.054123 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.054095 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/alertmanager/0.log" Jul 21 13:15:37.085558 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.085526 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/config-reloader/0.log" Jul 21 13:15:37.115229 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.115186 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/kube-rbac-proxy-web/0.log" Jul 21 13:15:37.139403 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.139379 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/kube-rbac-proxy/0.log" Jul 21 13:15:37.162949 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.162923 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/kube-rbac-proxy-metric/0.log" Jul 21 13:15:37.187058 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.187033 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/prom-label-proxy/0.log" Jul 21 13:15:37.211128 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.211046 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fe05a9ec-638a-4ecc-a849-1750cf15d8cb/init-config-reloader/0.log" Jul 21 13:15:37.294497 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.294470 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7822s_6866ac58-136c-4302-8088-7e911861c718/kube-state-metrics/0.log" Jul 21 13:15:37.319844 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.319820 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7822s_6866ac58-136c-4302-8088-7e911861c718/kube-rbac-proxy-main/0.log" Jul 21 13:15:37.347138 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.347115 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7822s_6866ac58-136c-4302-8088-7e911861c718/kube-rbac-proxy-self/0.log" Jul 21 13:15:37.378412 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.378383 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5ff668846c-jghsb_5e859268-78a0-464e-b08f-7f167f09ccd9/metrics-server/0.log" Jul 21 13:15:37.529945 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.529853 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdhc2_3a37cd6e-1ca8-415c-bfbb-7048bf5643fd/node-exporter/0.log" Jul 21 13:15:37.553771 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.553747 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdhc2_3a37cd6e-1ca8-415c-bfbb-7048bf5643fd/kube-rbac-proxy/0.log" Jul 21 13:15:37.589849 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.589820 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdhc2_3a37cd6e-1ca8-415c-bfbb-7048bf5643fd/init-textfile/0.log" Jul 21 13:15:37.743964 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.743934 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6mqs9_629adf2e-3a9b-4c51-bb75-dcaebc1c1156/kube-rbac-proxy-main/0.log" Jul 21 13:15:37.786331 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.786256 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6mqs9_629adf2e-3a9b-4c51-bb75-dcaebc1c1156/kube-rbac-proxy-self/0.log" Jul 21 13:15:37.831183 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.831153 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6mqs9_629adf2e-3a9b-4c51-bb75-dcaebc1c1156/openshift-state-metrics/0.log" Jul 21 13:15:37.898740 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.898713 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/prometheus/0.log" Jul 21 13:15:37.936931 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.936904 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/config-reloader/0.log" Jul 21 13:15:37.977437 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:37.977409 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/thanos-sidecar/0.log" Jul 21 13:15:38.014959 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.014935 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/kube-rbac-proxy-web/0.log" Jul 21 13:15:38.054112 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.054029 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/kube-rbac-proxy/0.log" Jul 21 13:15:38.090184 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.090157 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/kube-rbac-proxy-thanos/0.log" Jul 21 13:15:38.131908 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.131880 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4bd5500e-1af2-4448-8754-3da220096de0/init-config-reloader/0.log" Jul 21 13:15:38.179047 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.179010 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-8p6jf_e73ddc9e-fc8a-4612-87b4-e869b304199b/prometheus-operator/0.log" Jul 21 13:15:38.216981 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.216952 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-8p6jf_e73ddc9e-fc8a-4612-87b4-e869b304199b/kube-rbac-proxy/0.log" Jul 21 13:15:38.265280 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:38.265254 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-9xftz_2ed13308-0587-4739-b04c-042123d315d7/prometheus-operator-admission-webhook/0.log" Jul 21 13:15:39.411350 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411313 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl"] Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411678 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" containerName="test-curl-7509-139725296088240" Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411689 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" containerName="test-curl-7509-139725296088240" Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411708 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" containerName="main" Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411714 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" containerName="main" Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411815 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9f8f9e4-ad15-4a03-b3b1-20b87d7fabef" containerName="main" Jul 21 13:15:39.411896 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.411830 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9f04686-2b57-4d8a-9b9d-0e6f294a8179" containerName="test-curl-7509-139725296088240" Jul 21 13:15:39.418359 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.418103 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.421510 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.421481 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lpgz9\"/\"default-dockercfg-4r69j\"" Jul 21 13:15:39.422494 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.422469 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpgz9\"/\"openshift-service-ca.crt\"" Jul 21 13:15:39.422877 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.422471 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpgz9\"/\"kube-root-ca.crt\"" Jul 21 13:15:39.427112 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.427086 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl"] Jul 21 13:15:39.521704 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.521674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-sys\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.521704 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.521710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-lib-modules\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.521937 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.521817 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-podres\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.521937 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.521864 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4f2x\" (UniqueName: \"kubernetes.io/projected/2ca8d757-1685-4818-8ffa-7d7dc3958c15-kube-api-access-s4f2x\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.521937 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.521915 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-proc\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623417 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-podres\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623599 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623429 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4f2x\" (UniqueName: \"kubernetes.io/projected/2ca8d757-1685-4818-8ffa-7d7dc3958c15-kube-api-access-s4f2x\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623599 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-proc\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623678 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623602 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-proc\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623678 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623566 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-podres\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623749 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623660 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-sys\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623749 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623718 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-sys\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623749 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623724 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-lib-modules\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.623898 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.623882 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ca8d757-1685-4818-8ffa-7d7dc3958c15-lib-modules\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.632502 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.632455 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4f2x\" (UniqueName: \"kubernetes.io/projected/2ca8d757-1685-4818-8ffa-7d7dc3958c15-kube-api-access-s4f2x\") pod \"perf-node-gather-daemonset-mkwpl\" (UID: \"2ca8d757-1685-4818-8ffa-7d7dc3958c15\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.734992 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.734951 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:39.885867 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:39.885833 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl"] Jul 21 13:15:39.887725 ip-10-0-140-37 kubenswrapper[2584]: W0721 13:15:39.887700 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2ca8d757_1685_4818_8ffa_7d7dc3958c15.slice/crio-c5a37d1f99d5932964f2b57edb631c37c77fd39c062bca26833b8a66a3b12ceb WatchSource:0}: Error finding container c5a37d1f99d5932964f2b57edb631c37c77fd39c062bca26833b8a66a3b12ceb: Status 404 returned error can't find the container with id c5a37d1f99d5932964f2b57edb631c37c77fd39c062bca26833b8a66a3b12ceb Jul 21 13:15:40.220876 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.220832 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" event={"ID":"2ca8d757-1685-4818-8ffa-7d7dc3958c15","Type":"ContainerStarted","Data":"b4f4a4ef51483f38a2a31df806bbc53c03d8f7dddbb68a4cbc2973cbbf82e7df"} Jul 21 13:15:40.221055 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.220902 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" event={"ID":"2ca8d757-1685-4818-8ffa-7d7dc3958c15","Type":"ContainerStarted","Data":"c5a37d1f99d5932964f2b57edb631c37c77fd39c062bca26833b8a66a3b12ceb"} Jul 21 13:15:40.221055 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.221004 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:40.242045 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.241992 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" podStartSLOduration=1.241974646 podStartE2EDuration="1.241974646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:15:40.240371336 +0000 UTC m=+2887.763099052" watchObservedRunningTime="2026-07-21 13:15:40.241974646 +0000 UTC m=+2887.764702417" Jul 21 13:15:40.951672 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.951634 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76b54d755d-knnrx_bda00eb1-73a1-4a21-ad07-08567cd1cbcc/console/0.log" Jul 21 13:15:40.990044 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:40.990015 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-4xmlr_0f13f8cd-03f0-4664-8198-220f6bfb538b/download-server/0.log" Jul 21 13:15:42.710892 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:42.710866 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6svf_31e22981-1ca4-4f7e-be49-607fe638702a/dns/0.log" Jul 21 13:15:42.734503 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:42.734470 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6svf_31e22981-1ca4-4f7e-be49-607fe638702a/kube-rbac-proxy/0.log" Jul 21 13:15:42.812456 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:42.812430 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r27b5_f8b5a32b-9781-4981-bb66-3863b4ade135/dns-node-resolver/0.log" Jul 21 13:15:43.383569 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:43.383540 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7rvj2_e335be90-a7bf-4077-a72a-694dbddcb799/node-ca/0.log" Jul 21 13:15:44.619240 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:44.619201 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-hz7t2_fd3ef682-d12a-4ad9-a603-159f2969bd01/kube-auth-proxy/0.log" Jul 21 13:15:44.758463 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:44.758430 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/0.log" Jul 21 13:15:44.781764 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:44.781733 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/1.log" Jul 21 13:15:44.892028 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:44.891941 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c4984944-x74sv_0d642c82-dbe3-43f3-b083-9cb442147639/router/0.log" Jul 21 13:15:45.628097 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:45.628065 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4k5k_04fc799a-d832-4a53-9d8d-2a8bb58e0f47/serve-healthcheck-canary/0.log" Jul 21 13:15:46.111829 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:46.111801 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27cp7_6ffcfe75-064b-43de-a0ed-00b0414e0f3f/kube-rbac-proxy/0.log" Jul 21 13:15:46.138357 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:46.138325 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27cp7_6ffcfe75-064b-43de-a0ed-00b0414e0f3f/exporter/0.log" Jul 21 13:15:46.163920 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:46.163895 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27cp7_6ffcfe75-064b-43de-a0ed-00b0414e0f3f/extractor/0.log" Jul 21 13:15:46.237201 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:46.237173 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-mkwpl" Jul 21 13:15:49.334808 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:49.334775 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-hkrlt_4a6e2033-527e-4719-9ab8-e9576135c34e/manager/0.log" Jul 21 13:15:49.356904 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:49.356869 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-hkrlt_4a6e2033-527e-4719-9ab8-e9576135c34e/copy-manifests/0.log" Jul 21 13:15:50.115571 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:50.115541 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-hz7t2_fd3ef682-d12a-4ad9-a603-159f2969bd01/kube-auth-proxy/0.log" Jul 21 13:15:50.151838 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:50.151809 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/0.log" Jul 21 13:15:50.176390 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:50.176363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/1.log" Jul 21 13:15:50.241770 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:50.241743 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c4984944-x74sv_0d642c82-dbe3-43f3-b083-9cb442147639/router/0.log" Jul 21 13:15:50.679966 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:50.679940 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c45b47d79-rbfsp_66079a3f-aa83-4cd5-a13b-9422ebbcca34/manager/0.log" Jul 21 13:15:51.715037 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:51.714994 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-hz7t2_fd3ef682-d12a-4ad9-a603-159f2969bd01/kube-auth-proxy/0.log" Jul 21 13:15:51.753122 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:51.753077 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/0.log" Jul 21 13:15:51.777664 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:51.777626 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hx6tg_777c3364-4f85-42dd-a32d-0a102990cc5f/istio-proxy/1.log" Jul 21 13:15:51.848629 ip-10-0-140-37 kubenswrapper[2584]: I0721 13:15:51.848598 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c4984944-x74sv_0d642c82-dbe3-43f3-b083-9cb442147639/router/0.log"