Jul 17 13:45:13.134161 ip-10-0-133-123 systemd[1]: Starting Kubernetes Kubelet... Jul 17 13:45:13.486006 ip-10-0-133-123 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:45:13.486006 ip-10-0-133-123 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 13:45:13.486006 ip-10-0-133-123 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:45:13.486006 ip-10-0-133-123 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 13:45:13.486006 ip-10-0-133-123 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:45:13.486882 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.486739 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 13:45:13.492146 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492103 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.492349 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492325 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.492349 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492343 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.492349 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492353 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492362 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492368 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492373 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492378 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492383 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492388 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492392 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492397 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492401 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492406 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492410 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492420 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492424 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492471 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.492514 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492475 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492592 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492597 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492600 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492603 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492606 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492609 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492612 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492615 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492617 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492620 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492623 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492625 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492628 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492631 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492633 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492636 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492639 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492641 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492644 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.492885 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492647 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492649 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492652 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492654 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492657 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492660 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492662 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492665 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492671 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492673 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492676 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492679 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492681 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492686 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492692 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492695 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492698 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492700 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492705 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.493369 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492711 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492714 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492717 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492720 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492722 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492725 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492728 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492730 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492733 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492735 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492738 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492741 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492744 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492766 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492771 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492774 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492777 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492779 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492782 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492785 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.493871 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492788 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492790 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492793 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492796 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492798 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492801 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492804 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492807 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492810 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492815 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492818 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492821 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492824 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492827 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492830 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492834 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492837 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492841 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492844 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492847 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.494356 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492849 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.492852 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493600 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493608 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493611 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493614 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493618 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493622 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493626 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493630 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493633 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493637 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493640 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493643 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493646 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493649 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493652 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493655 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493657 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493661 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.494874 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493664 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493667 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493670 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493673 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493675 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493678 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493680 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493683 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493685 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493689 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493693 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493696 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493698 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493701 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493703 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493706 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493710 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493713 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493718 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.495412 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493721 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493724 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493727 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493730 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493733 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493736 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493739 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493741 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493744 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493763 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493766 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493769 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493772 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493776 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493779 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493782 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493785 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493787 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493790 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.495920 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493793 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493796 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493798 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493801 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493803 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493806 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493808 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493811 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493814 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493817 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493819 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493822 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493825 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493828 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493831 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493837 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493839 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493842 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493844 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493847 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.496393 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493850 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493852 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493855 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493858 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493860 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493863 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493865 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493868 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493871 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493874 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493876 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493879 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493882 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493884 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493887 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493889 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493892 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493894 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493897 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493899 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.496896 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493904 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493906 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.493909 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494320 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494325 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494328 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494331 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494335 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494338 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494341 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494344 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494347 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494349 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494352 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494355 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494358 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494360 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494364 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494366 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494370 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.497395 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494373 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494376 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494378 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494381 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494383 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494386 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494388 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494391 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494393 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494396 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494398 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494401 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494406 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494409 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494411 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494414 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494417 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494419 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494422 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494425 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.497888 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494428 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494431 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494433 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494436 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494438 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494441 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494443 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494446 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494448 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494453 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494456 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494459 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494467 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494470 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494473 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494476 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494479 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494481 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494484 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.498374 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494487 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494490 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494493 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494495 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494498 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494502 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494505 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494508 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494512 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494514 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494517 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494519 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494531 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494535 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494538 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494541 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494544 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494546 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494549 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.498855 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494551 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494554 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494556 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494559 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494562 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494564 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494567 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494570 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494573 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494575 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494577 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494580 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494582 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494585 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494587 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494590 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494592 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494595 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494599 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494602 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.499312 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494604 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494607 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494609 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.494612 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494690 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494700 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494708 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494712 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494717 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494720 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494724 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494729 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494732 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494735 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494738 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494742 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494763 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494768 2555 flags.go:64] FLAG: --cgroup-root="" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494773 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494778 2555 flags.go:64] FLAG: --client-ca-file="" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494783 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494786 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494790 2555 flags.go:64] FLAG: --cluster-domain="" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494793 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 13:45:13.499809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494797 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494800 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494803 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494807 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494810 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494813 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494818 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494821 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494824 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494828 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494831 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494834 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494838 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494843 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494846 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494849 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494852 2555 flags.go:64] FLAG: --enable-server="true" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494855 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494860 2555 flags.go:64] FLAG: --event-burst="100" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494863 2555 flags.go:64] FLAG: --event-qps="50" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494866 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494869 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494872 2555 flags.go:64] FLAG: --eviction-hard="" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494876 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 13:45:13.500372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494879 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494882 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494885 2555 flags.go:64] FLAG: --eviction-soft="" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494888 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494891 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494895 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494898 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494901 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494903 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494906 2555 flags.go:64] FLAG: --feature-gates="" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494910 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494913 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494917 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494920 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494924 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494927 2555 flags.go:64] FLAG: --help="false" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494930 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494933 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494936 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494939 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494943 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494947 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494950 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 13:45:13.500986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494953 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494956 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494959 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494962 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494966 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494969 2555 flags.go:64] FLAG: --kube-reserved="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494971 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494974 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494977 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494980 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494983 2555 flags.go:64] FLAG: --lock-file="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494986 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494989 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494992 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.494997 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495001 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495004 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495007 2555 flags.go:64] FLAG: --logging-format="text" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495010 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495013 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495016 2555 flags.go:64] FLAG: --manifest-url="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495019 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495023 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495027 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495032 2555 flags.go:64] FLAG: --max-pods="110" Jul 17 13:45:13.501577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495034 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495037 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495040 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495043 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495046 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495051 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495054 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495061 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495065 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495068 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495071 2555 flags.go:64] FLAG: --pod-cidr="" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495074 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495079 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495082 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495085 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495088 2555 flags.go:64] FLAG: --port="10250" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495091 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495094 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05495471c36dc467b" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495097 2555 flags.go:64] FLAG: --qos-reserved="" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495100 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495103 2555 flags.go:64] FLAG: --register-node="true" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495106 2555 flags.go:64] FLAG: --register-with-taints="" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495110 2555 flags.go:64] FLAG: --registry-burst="10" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495114 2555 flags.go:64] FLAG: --registry-qps="5" Jul 17 13:45:13.502216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495117 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495120 2555 flags.go:64] FLAG: --reserved-memory="" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495123 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495126 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495129 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495132 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495135 2555 flags.go:64] FLAG: --runonce="false" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495139 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495142 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495145 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495148 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495151 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495154 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495159 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495162 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495165 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495168 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495171 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495174 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495177 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495180 2555 flags.go:64] FLAG: --system-cgroups="" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495183 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495188 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495191 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495194 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 13:45:13.502838 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495198 2555 flags.go:64] FLAG: --tls-min-version="" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495200 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495203 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495206 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495209 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495212 2555 flags.go:64] FLAG: --v="2" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495216 2555 flags.go:64] FLAG: --version="false" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495221 2555 flags.go:64] FLAG: --vmodule="" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495225 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495228 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495355 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495359 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495363 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495365 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495370 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495372 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495375 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495378 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495380 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495383 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495388 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495390 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.503508 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495393 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495395 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495399 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495402 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495404 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495407 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495409 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495412 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495416 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495420 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495423 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495425 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495428 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495431 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495434 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495436 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495439 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495441 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495444 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495447 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.504071 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495450 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495452 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495454 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495457 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495460 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495463 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495465 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495468 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495471 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495473 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495478 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495480 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495483 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495485 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495488 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495491 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495493 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495496 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495498 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495502 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.504592 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495506 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495509 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495512 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495515 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495517 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495520 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495523 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495526 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495529 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495532 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495534 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495537 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495540 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495543 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495546 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495549 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495551 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495554 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495556 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.505103 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495559 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495561 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495564 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495569 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495572 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495575 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495579 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495582 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495584 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495587 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495590 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495592 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495595 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495597 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495600 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495602 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495605 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495608 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495610 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495612 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.505578 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495615 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495617 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495620 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495622 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495625 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495628 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495631 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.495633 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.495639 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503048 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503072 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503088 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503094 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503158 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.506081 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503166 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503169 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503173 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503176 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503178 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503181 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503185 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503188 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503191 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503194 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503197 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503200 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503203 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503206 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503208 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503211 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503214 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503217 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503220 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503223 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.506434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503225 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503228 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503230 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503233 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503236 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503239 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503241 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503244 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503255 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503258 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503263 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503267 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503270 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503273 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503276 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503280 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503284 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503287 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503290 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.506942 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503293 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503296 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503299 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503301 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503304 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503307 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503309 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503312 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503315 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503318 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503320 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503323 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503325 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503328 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503331 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503333 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503336 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503339 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503341 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503343 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.507408 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503346 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503350 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503357 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503360 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503362 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503365 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503368 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503370 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503373 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503376 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503379 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503381 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503384 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503387 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503390 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503392 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503395 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503397 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503400 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503403 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.507911 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503405 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503408 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503410 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503412 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503415 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503417 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503420 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503423 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503426 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503428 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503430 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503434 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503436 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503439 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503442 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503445 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503448 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503450 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.508394 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503453 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503458 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503600 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503605 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503621 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503625 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503628 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503631 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503634 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503637 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503639 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503642 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503645 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503647 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503650 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:45:13.508847 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503653 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503656 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503658 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503661 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503663 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503666 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503669 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503671 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503674 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503677 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503680 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503683 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503686 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503689 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503691 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503695 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503698 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503701 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503703 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503706 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:45:13.509214 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503709 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503711 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503714 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503716 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503719 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503721 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503724 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503727 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503729 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503732 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503734 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503737 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503739 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503742 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503745 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503764 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503767 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503770 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503773 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503775 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:45:13.509709 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503778 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503781 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503784 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503787 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503789 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503792 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503797 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503800 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503804 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503807 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503810 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503813 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503816 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503819 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503822 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503824 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503827 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503830 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503834 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:45:13.510210 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503837 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503840 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503843 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503846 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503848 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503851 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503853 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503857 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503859 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503862 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503864 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503867 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503870 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503872 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503875 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503878 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503881 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503883 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503886 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:45:13.510674 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503889 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503893 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503896 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503900 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503903 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503906 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503908 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:13.503911 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.503916 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.504089 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.506989 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.508338 2555 server.go:1019] "Starting client certificate rotation" Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.508447 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 13:45:13.511165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.509097 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 13:45:13.511495 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.510831 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 13:45:13.512049 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.512015 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 13:45:13.522978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.522949 2555 log.go:25] "Validated CRI v1 runtime API" Jul 17 13:45:13.525493 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.525472 2555 log.go:25] "Validated CRI v1 image API" Jul 17 13:45:13.526456 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.526438 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 13:45:13.530187 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.530163 2555 fs.go:135] Filesystem UUIDs: map[0a11a6dd-df52-48b3-9100-2e46a2f88e71:/dev/nvme0n1p4 47b489c4-9f7c-4b6a-a25e-af5d5bba1567:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 17 13:45:13.530283 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.530185 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 13:45:13.535830 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.535674 2555 manager.go:216] Machine: {Timestamp:2026-07-17 13:45:13.534284109 +0000 UTC m=+0.295149112 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099606 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c76222266ba7676d987111cbdfbb5 SystemUUID:ec2c7622-2266-ba76-76d9-87111cbdfbb5 BootID:15aa10db-db92-4a78-baf2-711579f87a11 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6f:7c:ce:ac:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6f:7c:ce:ac:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:bf:f9:3e:4c:f5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 13:45:13.535830 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.535819 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 13:45:13.535968 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.535920 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 13:45:13.539041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.539008 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 13:45:13.539210 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.539046 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 13:45:13.540324 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.540310 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 13:45:13.540358 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.540328 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 13:45:13.540358 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.540336 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 13:45:13.540963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.540952 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 13:45:13.542386 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.542373 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 13:45:13.546624 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546606 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 17 13:45:13.546801 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546788 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 13:45:13.546856 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546827 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 17 13:45:13.546856 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546843 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 13:45:13.546950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546867 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 13:45:13.546950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546881 2555 kubelet.go:398] "Adding apiserver pod source" Jul 17 13:45:13.546950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.546891 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 13:45:13.547072 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.547032 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 13:45:13.547352 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.547339 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 13:45:13.547405 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.547358 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 13:45:13.549949 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.549924 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 13:45:13.552263 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.552229 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 13:45:13.552349 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.552289 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 13:45:13.553143 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553119 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553157 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553167 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553176 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553188 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553197 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553209 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553217 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553228 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 13:45:13.553251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553246 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 13:45:13.553619 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553265 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 13:45:13.553619 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.553276 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 13:45:13.554109 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.554090 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 13:45:13.554188 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.554121 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 13:45:13.557805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.557781 2555 server.go:1268] "Started kubelet" Jul 17 13:45:13.557927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.557855 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 13:45:13.558164 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.558090 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 13:45:13.558164 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.558157 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 13:45:13.559385 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.559362 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 13:45:13.559379 ip-10-0-133-123 systemd[1]: Started Kubernetes Kubelet. Jul 17 13:45:13.559678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.559532 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 13:45:13.559678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.559642 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 17 13:45:13.560294 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.560260 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 13:45:13.560383 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.560239 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 13:45:13.563226 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.563204 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 13:45:13.566949 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.566926 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 13:45:13.566949 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.566926 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 13:45:13.567129 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.567047 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:13.567576 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.567560 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 13:45:13.567706 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.567694 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 13:45:13.567802 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.567793 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 13:45:13.568005 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.567994 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 17 13:45:13.568087 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.568078 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 17 13:45:13.568957 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.568907 2555 factory.go:153] Registering CRI-O factory Jul 17 13:45:13.568957 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.568927 2555 factory.go:223] Registration of the crio container factory successfully Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.568987 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.568997 2555 factory.go:55] Registering systemd factory Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.569005 2555 factory.go:223] Registration of the systemd container factory successfully Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.569032 2555 factory.go:103] Registering Raw factory Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.569049 2555 manager.go:1196] Started watching for new ooms in manager Jul 17 13:45:13.569108 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.569065 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 13:45:13.569469 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.569447 2555 manager.go:349] Starting recovery of all containers Jul 17 13:45:13.570014 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.569096 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18c317ae05ca1afe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-07-17 13:45:13.557727998 +0000 UTC m=+0.318593002,LastTimestamp:2026-07-17 13:45:13.557727998 +0000 UTC m=+0.318593002,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}" Jul 17 13:45:13.572329 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.572293 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 13:45:13.580871 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.580848 2555 manager.go:354] Recovery completed Jul 17 13:45:13.585403 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.585389 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.587302 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587278 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.587363 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587320 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.587363 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587332 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.587860 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587847 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 13:45:13.587860 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587860 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 13:45:13.587963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587872 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.587963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.587878 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 17 13:45:13.588541 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.588527 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.588580 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.588553 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.588580 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.588562 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.589512 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.589447 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18c317ae078d6a38 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-07-17 13:45:13.587305016 +0000 UTC m=+0.348170023,LastTimestamp:2026-07-17 13:45:13.587305016 +0000 UTC m=+0.348170023,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}" Jul 17 13:45:13.590126 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.590115 2555 policy_none.go:49] "None policy: Start" Jul 17 13:45:13.590160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.590134 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 13:45:13.590160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.590145 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 13:45:13.592131 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.592118 2555 policy_none.go:47] "Start" Jul 17 13:45:13.603129 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.603055 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18c317ae078dbca9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-123.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-07-17 13:45:13.587326121 +0000 UTC m=+0.348191125,LastTimestamp:2026-07-17 13:45:13.587326121 +0000 UTC m=+0.348191125,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}" Jul 17 13:45:13.612556 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.612347 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9tm2j" Jul 17 13:45:13.614189 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.614089 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18c317ae078de33d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-123.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-07-17 13:45:13.587335997 +0000 UTC m=+0.348201002,LastTimestamp:2026-07-17 13:45:13.587335997 +0000 UTC m=+0.348201002,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}" Jul 17 13:45:13.618260 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.618240 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9tm2j" Jul 17 13:45:13.623139 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.623034 2555 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-123.ec2.internal.18c317ae078d6a38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-123.ec2.internal.18c317ae078d6a38 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-123.ec2.internal,UID:ip-10-0-133-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-123.ec2.internal,},FirstTimestamp:2026-07-17 13:45:13.587305016 +0000 UTC m=+0.348170023,LastTimestamp:2026-07-17 13:45:13.588540709 +0000 UTC m=+0.349405713,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-123.ec2.internal,}" Jul 17 13:45:13.629533 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.629516 2555 manager.go:337] "Starting Device Plugin manager" Jul 17 13:45:13.629627 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.629552 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 13:45:13.629627 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.629563 2555 server.go:87] "Starting device plugin registration server" Jul 17 13:45:13.629894 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.629879 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 13:45:13.629941 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.629896 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 13:45:13.630014 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.629996 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 13:45:13.630113 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.630096 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 13:45:13.630113 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.630109 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 13:45:13.630713 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.630694 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 13:45:13.630796 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.630743 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:13.671086 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.671041 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 13:45:13.677068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.672314 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 13:45:13.677068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.672350 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 13:45:13.677068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.672373 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 13:45:13.677068 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.672411 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 13:45:13.677068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.674965 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:45:13.732801 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.732734 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.733857 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.733836 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.733965 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.733872 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.733965 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.733883 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.733965 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.733916 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.744405 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.744338 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.744405 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.744364 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-123.ec2.internal\": node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:13.773369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.773338 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal"] Jul 17 13:45:13.773559 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.773414 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.775125 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.775107 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.775199 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.775139 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.775199 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.775150 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.777686 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.777673 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.777844 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.777818 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.777946 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.777858 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.778493 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778477 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.778493 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778491 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.778614 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778509 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.778614 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778512 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.778614 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778548 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.778614 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.778525 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.781086 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.781071 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.781134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.781098 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:13.782357 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.782339 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:13.782428 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.782370 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:13.782428 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.782384 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:13.784404 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.784382 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:13.799506 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.799482 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-123.ec2.internal\" not found" node="ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.806282 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.806260 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-123.ec2.internal\" not found" node="ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.870947 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.870910 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.871111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.870954 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.871111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.870981 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f128f886475b532dbbbd78614ba32f76-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"f128f886475b532dbbbd78614ba32f76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.884551 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.884518 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:13.971959 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.971927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.971959 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.971960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.972213 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.971982 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f128f886475b532dbbbd78614ba32f76-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"f128f886475b532dbbbd78614ba32f76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.972213 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.972025 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f128f886475b532dbbbd78614ba32f76-config\") pod \"kube-apiserver-proxy-ip-10-0-133-123.ec2.internal\" (UID: \"f128f886475b532dbbbd78614ba32f76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.972213 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.972045 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.972213 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:13.972056 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae320061f58d9bc26e8732459e276d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal\" (UID: \"99ae320061f58d9bc26e8732459e276d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:13.984992 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:13.984966 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.085175 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.085089 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.102119 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.102075 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:14.110925 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.110907 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:14.186362 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.186313 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.286770 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.286733 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.387304 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.387220 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.488290 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.488237 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.507873 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.507832 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 13:45:14.508050 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.508027 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 13:45:14.564877 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.564844 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:45:14.588834 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.588805 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.624036 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.623989 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 13:40:13 +0000 UTC" deadline="2028-01-08 14:20:07.985514742 +0000 UTC" Jul 17 13:45:14.624036 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.624030 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12960h34m53.361488518s" Jul 17 13:45:14.631836 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.631791 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:45:14.633062 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.633042 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:45:14.633174 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.633080 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:45:14.633174 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.633098 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:45:14.689249 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.689166 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.790221 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.790180 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.841550 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:14.841516 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ae320061f58d9bc26e8732459e276d.slice/crio-47d01989ccd1cc5da3226252629a2b1e086edabd374a2b07d787a99a5d56ab8a WatchSource:0}: Error finding container 47d01989ccd1cc5da3226252629a2b1e086edabd374a2b07d787a99a5d56ab8a: Status 404 returned error can't find the container with id 47d01989ccd1cc5da3226252629a2b1e086edabd374a2b07d787a99a5d56ab8a Jul 17 13:45:14.841876 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:14.841851 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf128f886475b532dbbbd78614ba32f76.slice/crio-370f9175225c7b151b147b7d090d752c518deac70af161b50a0c397a71858676 WatchSource:0}: Error finding container 370f9175225c7b151b147b7d090d752c518deac70af161b50a0c397a71858676: Status 404 returned error can't find the container with id 370f9175225c7b151b147b7d090d752c518deac70af161b50a0c397a71858676 Jul 17 13:45:14.847948 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.847932 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:45:14.852162 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.852141 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:45:14.890712 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:14.890650 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-123.ec2.internal\" not found" Jul 17 13:45:14.895417 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.895397 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:45:14.972573 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.972492 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" Jul 17 13:45:14.983301 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.983275 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 13:45:14.984425 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.984404 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" Jul 17 13:45:14.998051 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:14.998022 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 13:45:15.549875 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.549822 2555 apiserver.go:52] "Watching apiserver" Jul 17 13:45:15.556093 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.556067 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:45:15.556641 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.556620 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 13:45:15.558805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.558783 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-r9xs2","kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal","openshift-dns/node-resolver-rtgps","openshift-multus/multus-additional-cni-plugins-xbpph","openshift-multus/multus-v86dk","openshift-multus/network-metrics-daemon-v2nz4","openshift-network-operator/iptables-alerter-t7vhz","openshift-ovn-kubernetes/ovnkube-node-ltngj","kube-system/konnectivity-agent-2n5dq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6","openshift-cluster-node-tuning-operator/tuned-lxqzv","openshift-image-registry/node-ca-4bfpp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal","openshift-network-diagnostics/network-check-target-8kgqr"] Jul 17 13:45:15.562809 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.562791 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.562879 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.562864 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:15.565696 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.565679 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.567632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.567606 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.568110 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.569003 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.569327 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.569482 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.569697 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.569976 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.570115 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 13:45:15.570318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.570215 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fvrtp\"" Jul 17 13:45:15.572308 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.572290 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.573581 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.573560 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.573678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.573614 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9l7m4\"" Jul 17 13:45:15.573678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.573652 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.573797 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.573693 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 13:45:15.574551 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.574530 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94xm6\"" Jul 17 13:45:15.574649 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.574585 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.576452 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576371 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 13:45:15.576452 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576369 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 13:45:15.576452 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576374 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.576656 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576556 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.576656 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576569 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qqgc7\"" Jul 17 13:45:15.576656 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.576597 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 13:45:15.577041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.577022 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 13:45:15.577137 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.577040 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.579858 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.579841 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.579943 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.579911 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:15.582146 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-netd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582146 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582115 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-config\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582146 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582140 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-env-overrides\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582157 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/703ae377-e2ed-4493-957d-7fa6599ee17c-host-slash\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmps\" (UniqueName: \"kubernetes.io/projected/33042295-c3b5-46b6-bdb9-7d6c9095f83f-kube-api-access-tnmps\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582296 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-kubelet\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-slash\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582351 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-systemd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-var-lib-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582416 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-etc-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582433 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-log-socket\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582495 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582526 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-systemd-units\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582563 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582591 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f8757ff-c282-4729-b550-202d7a617d91-ovn-node-metrics-cert\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-script-lib\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.582692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582673 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6xnm\" (UniqueName: \"kubernetes.io/projected/6f8757ff-c282-4729-b550-202d7a617d91-kube-api-access-t6xnm\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582722 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/703ae377-e2ed-4493-957d-7fa6599ee17c-iptables-alerter-script\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582786 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-bin\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582825 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrjcj\" (UniqueName: \"kubernetes.io/projected/703ae377-e2ed-4493-957d-7fa6599ee17c-kube-api-access-nrjcj\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-netns\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-ovn\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583190 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.582919 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-node-log\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.583930 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.583747 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.585105 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.584941 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kvc82\"" Jul 17 13:45:15.585105 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.584956 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.585305 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.585205 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 13:45:15.585305 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.585240 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.585419 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.585403 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkr9q\"" Jul 17 13:45:15.585768 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.585734 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 13:45:15.587957 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.587937 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.590711 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.590688 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.591342 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591105 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.591342 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591111 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 13:45:15.591342 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591110 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.591549 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591445 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cvf66\"" Jul 17 13:45:15.591549 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591460 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.591655 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591591 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wjsrq\"" Jul 17 13:45:15.591711 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591671 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 13:45:15.591997 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.591978 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.593384 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.593359 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:15.593500 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.593428 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:15.600478 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.600451 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 13:45:15.600816 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.600794 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 13:45:15.600816 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.600822 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 13:45:15.601037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.600916 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xd7qw\"" Jul 17 13:45:15.638098 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.638056 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6j4cj" Jul 17 13:45:15.647820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.647540 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6j4cj" Jul 17 13:45:15.669068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.669034 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 13:45:15.678999 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.678940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"99ae320061f58d9bc26e8732459e276d","Type":"ContainerStarted","Data":"47d01989ccd1cc5da3226252629a2b1e086edabd374a2b07d787a99a5d56ab8a"} Jul 17 13:45:15.680593 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.680561 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" event={"ID":"f128f886475b532dbbbd78614ba32f76","Type":"ContainerStarted","Data":"370f9175225c7b151b147b7d090d752c518deac70af161b50a0c397a71858676"} Jul 17 13:45:15.683470 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/703ae377-e2ed-4493-957d-7fa6599ee17c-host-slash\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.683585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683485 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-etc-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.683585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-log-socket\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.683585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683538 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6xnm\" (UniqueName: \"kubernetes.io/projected/6f8757ff-c282-4729-b550-202d7a617d91-kube-api-access-t6xnm\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.683585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683569 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-k8s-cni-cncf-io\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683575 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/703ae377-e2ed-4493-957d-7fa6599ee17c-host-slash\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683596 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-etc-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-kubelet\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683594 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-log-socket\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181882a0-faee-4bd6-bce1-e4baf72269f8-tmp-dir\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683704 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp9tf\" (UniqueName: \"kubernetes.io/projected/5055a98d-0e09-468e-a2f8-7cf602f1df79-kube-api-access-kp9tf\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.683757 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683768 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683806 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-script-lib\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683823 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-system-cni-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683862 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzmn4\" (UniqueName: \"kubernetes.io/projected/012f0baa-3293-439e-b169-4400362a0857-kube-api-access-vzmn4\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683944 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-socket-dir-parent\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.683992 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-kubernetes\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-var-lib-kubelet\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-netd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684066 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.684117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684111 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-netd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684145 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-device-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684172 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-netns\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-bin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684221 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjdkf\" (UniqueName: \"kubernetes.io/projected/3517e45a-1fe0-434f-b546-ef3db5198201-kube-api-access-cjdkf\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-systemd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-var-lib-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684356 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f8757ff-c282-4729-b550-202d7a617d91-ovn-node-metrics-cert\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684360 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-systemd\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684382 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/703ae377-e2ed-4493-957d-7fa6599ee17c-iptables-alerter-script\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684383 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-var-lib-openvswitch\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684400 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684408 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-conf-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684431 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/181882a0-faee-4bd6-bce1-e4baf72269f8-hosts-file\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684458 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684480 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-systemd-units\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.684847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-script-lib\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684505 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684530 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-cnibin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684552 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-os-release\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-bin\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684605 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-systemd-units\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-cnibin\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684649 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684681 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-multus-certs\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.684719 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684740 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbcefb70-7a53-4e68-ad3f-6411e521bef8-konnectivity-ca\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684778 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-cni-bin\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.684823 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:16.184802366 +0000 UTC m=+2.945667373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-systemd\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-netns\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.685732 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684890 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-ovn\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-env-overrides\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684914 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/703ae377-e2ed-4493-957d-7fa6599ee17c-iptables-alerter-script\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684940 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-kubelet-config\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684973 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.684983 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-run-ovn\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-conf\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685021 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-run-netns\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685047 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-host\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685068 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-tmp\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmps\" (UniqueName: \"kubernetes.io/projected/33042295-c3b5-46b6-bdb9-7d6c9095f83f-kube-api-access-tnmps\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685186 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-slash\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-etc-kubernetes\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-slash\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-run\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685313 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-sys\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.686511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685351 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-sys-fs\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685370 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-modprobe-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685395 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-lib-modules\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685421 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrgvp\" (UniqueName: \"kubernetes.io/projected/181882a0-faee-4bd6-bce1-e4baf72269f8-kube-api-access-wrgvp\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685445 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdpch\" (UniqueName: \"kubernetes.io/projected/6a850987-4fc3-45b7-98a7-8fa8b483573a-kube-api-access-gdpch\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685471 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c56b3af1-0414-43b6-a26f-1c8c7ab26463-host\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685444 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-env-overrides\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c56b3af1-0414-43b6-a26f-1c8c7ab26463-serviceca\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685536 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-config\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685561 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-multus\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685596 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbcefb70-7a53-4e68-ad3f-6411e521bef8-agent-certs\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685717 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-registration-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685748 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685788 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht572\" (UniqueName: \"kubernetes.io/projected/c56b3af1-0414-43b6-a26f-1c8c7ab26463-kube-api-access-ht572\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.687303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685818 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrjcj\" (UniqueName: \"kubernetes.io/projected/703ae377-e2ed-4493-957d-7fa6599ee17c-kube-api-access-nrjcj\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-kubelet\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685868 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-os-release\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-system-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685918 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-cni-binary-copy\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685917 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-host-kubelet\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685942 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-dbus\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.685980 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-socket-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686022 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-multus-daemon-config\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686047 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysconfig\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686071 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-tuned\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686079 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f8757ff-c282-4729-b550-202d7a617d91-ovnkube-config\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-hostroot\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686144 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-node-log\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.688057 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.686193 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f8757ff-c282-4729-b550-202d7a617d91-node-log\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.689232 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.689210 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f8757ff-c282-4729-b550-202d7a617d91-ovn-node-metrics-cert\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.696126 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.696102 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6xnm\" (UniqueName: \"kubernetes.io/projected/6f8757ff-c282-4729-b550-202d7a617d91-kube-api-access-t6xnm\") pod \"ovnkube-node-ltngj\" (UID: \"6f8757ff-c282-4729-b550-202d7a617d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.700508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.700441 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmps\" (UniqueName: \"kubernetes.io/projected/33042295-c3b5-46b6-bdb9-7d6c9095f83f-kube-api-access-tnmps\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:15.701790 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.701766 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrjcj\" (UniqueName: \"kubernetes.io/projected/703ae377-e2ed-4493-957d-7fa6599ee17c-kube-api-access-nrjcj\") pod \"iptables-alerter-t7vhz\" (UID: \"703ae377-e2ed-4493-957d-7fa6599ee17c\") " pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.786489 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786453 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-run\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786489 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-sys\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786549 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-sys-fs\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786573 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-modprobe-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786617 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-lib-modules\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786619 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrgvp\" (UniqueName: \"kubernetes.io/projected/181882a0-faee-4bd6-bce1-e4baf72269f8-kube-api-access-wrgvp\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786619 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-sys\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786667 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-sys-fs\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786666 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-run\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786689 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdpch\" (UniqueName: \"kubernetes.io/projected/6a850987-4fc3-45b7-98a7-8fa8b483573a-kube-api-access-gdpch\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786715 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c56b3af1-0414-43b6-a26f-1c8c7ab26463-host\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786741 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c56b3af1-0414-43b6-a26f-1c8c7ab26463-serviceca\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.786742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786768 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-multus\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786782 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-modprobe-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786783 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-lib-modules\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbcefb70-7a53-4e68-ad3f-6411e521bef8-agent-certs\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786821 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c56b3af1-0414-43b6-a26f-1c8c7ab26463-host\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786881 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-multus\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786915 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-registration-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.786931 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.786960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht572\" (UniqueName: \"kubernetes.io/projected/c56b3af1-0414-43b6-a26f-1c8c7ab26463-kube-api-access-ht572\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.786990 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:16.286968898 +0000 UTC m=+3.047833895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-os-release\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-system-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787064 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-cni-binary-copy\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-dbus\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-socket-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.787440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-multus-daemon-config\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysconfig\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787184 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-tuned\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787212 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-hostroot\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-registration-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787245 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-k8s-cni-cncf-io\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-kubelet\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787290 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-k8s-cni-cncf-io\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181882a0-faee-4bd6-bce1-e4baf72269f8-tmp-dir\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp9tf\" (UniqueName: \"kubernetes.io/projected/5055a98d-0e09-468e-a2f8-7cf602f1df79-kube-api-access-kp9tf\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-system-cni-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-os-release\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787392 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmn4\" (UniqueName: \"kubernetes.io/projected/012f0baa-3293-439e-b169-4400362a0857-kube-api-access-vzmn4\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-socket-dir-parent\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-socket-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787486 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-kubelet\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787595 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c56b3af1-0414-43b6-a26f-1c8c7ab26463-serviceca\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.788047 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787711 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-system-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787779 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181882a0-faee-4bd6-bce1-e4baf72269f8-tmp-dir\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788015 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-multus-daemon-config\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788069 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-system-cni-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788083 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-hostroot\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788137 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-socket-dir-parent\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysconfig\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788193 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3517e45a-1fe0-434f-b546-ef3db5198201-cni-binary-copy\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.787449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788243 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-kubernetes\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-var-lib-kubelet\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788299 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-dbus\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-device-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788323 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-kubernetes\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-netns\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788349 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-var-lib-kubelet\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.788774 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788368 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-bin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjdkf\" (UniqueName: \"kubernetes.io/projected/3517e45a-1fe0-434f-b546-ef3db5198201-kube-api-access-cjdkf\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788423 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-netns\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788426 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-conf-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788461 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-conf-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/181882a0-faee-4bd6-bce1-e4baf72269f8-hosts-file\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788468 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788393 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5055a98d-0e09-468e-a2f8-7cf602f1df79-device-dir\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788500 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/181882a0-faee-4bd6-bce1-e4baf72269f8-hosts-file\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788525 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788536 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-var-lib-cni-bin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788553 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-cnibin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788579 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-os-release\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788638 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-cnibin\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788657 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-cnibin\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-os-release\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.789700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788752 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012f0baa-3293-439e-b169-4400362a0857-cnibin\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788756 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-multus-certs\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbcefb70-7a53-4e68-ad3f-6411e521bef8-konnectivity-ca\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-multus-cni-dir\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788850 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-systemd\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788879 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-kubelet-config\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788904 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788930 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-conf\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.788957 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-host\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789016 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-tmp\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789048 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789057 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789077 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-etc-kubernetes\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789146 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-etc-kubernetes\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-host\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789193 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-conf\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789243 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-systemd\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.790391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789249 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789289 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58a9967a-28c3-45cd-af44-13d2619266fc-kubelet-config\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3517e45a-1fe0-434f-b546-ef3db5198201-host-run-multus-certs\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789384 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbcefb70-7a53-4e68-ad3f-6411e521bef8-konnectivity-ca\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789392 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-sysctl-d\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.789763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012f0baa-3293-439e-b169-4400362a0857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.791287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.790489 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-etc-tuned\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.791978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.791953 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbcefb70-7a53-4e68-ad3f-6411e521bef8-agent-certs\") pod \"konnectivity-agent-2n5dq\" (UID: \"fbcefb70-7a53-4e68-ad3f-6411e521bef8\") " pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.792089 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.792007 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a850987-4fc3-45b7-98a7-8fa8b483573a-tmp\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.805865 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.805783 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:15.805865 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.805830 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:15.805865 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.805847 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:15.806048 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:15.805942 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:16.305919373 +0000 UTC m=+3.066784368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:15.806626 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.806598 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht572\" (UniqueName: \"kubernetes.io/projected/c56b3af1-0414-43b6-a26f-1c8c7ab26463-kube-api-access-ht572\") pod \"node-ca-4bfpp\" (UID: \"c56b3af1-0414-43b6-a26f-1c8c7ab26463\") " pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.809601 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.809387 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrgvp\" (UniqueName: \"kubernetes.io/projected/181882a0-faee-4bd6-bce1-e4baf72269f8-kube-api-access-wrgvp\") pod \"node-resolver-rtgps\" (UID: \"181882a0-faee-4bd6-bce1-e4baf72269f8\") " pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.811172 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.811117 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdpch\" (UniqueName: \"kubernetes.io/projected/6a850987-4fc3-45b7-98a7-8fa8b483573a-kube-api-access-gdpch\") pod \"tuned-lxqzv\" (UID: \"6a850987-4fc3-45b7-98a7-8fa8b483573a\") " pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.812427 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.812404 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjdkf\" (UniqueName: \"kubernetes.io/projected/3517e45a-1fe0-434f-b546-ef3db5198201-kube-api-access-cjdkf\") pod \"multus-v86dk\" (UID: \"3517e45a-1fe0-434f-b546-ef3db5198201\") " pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.813524 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.813501 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp9tf\" (UniqueName: \"kubernetes.io/projected/5055a98d-0e09-468e-a2f8-7cf602f1df79-kube-api-access-kp9tf\") pod \"aws-ebs-csi-driver-node-ghfv6\" (UID: \"5055a98d-0e09-468e-a2f8-7cf602f1df79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.814030 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.814007 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzmn4\" (UniqueName: \"kubernetes.io/projected/012f0baa-3293-439e-b169-4400362a0857-kube-api-access-vzmn4\") pod \"multus-additional-cni-plugins-xbpph\" (UID: \"012f0baa-3293-439e-b169-4400362a0857\") " pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.879582 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.879538 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:15.889661 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.889627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7vhz" Jul 17 13:45:15.891127 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.891102 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8757ff_c282_4729_b550_202d7a617d91.slice/crio-b3bdd909b987d8f61312a166ba7134db36214aeffca2751e21d3a27327a5a9e2 WatchSource:0}: Error finding container b3bdd909b987d8f61312a166ba7134db36214aeffca2751e21d3a27327a5a9e2: Status 404 returned error can't find the container with id b3bdd909b987d8f61312a166ba7134db36214aeffca2751e21d3a27327a5a9e2 Jul 17 13:45:15.899526 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.899497 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbpph" Jul 17 13:45:15.899649 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.899622 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703ae377_e2ed_4493_957d_7fa6599ee17c.slice/crio-36f06475b258b116ecdb479aaaad602af29d9ba76110b29e6010cef059dff17b WatchSource:0}: Error finding container 36f06475b258b116ecdb479aaaad602af29d9ba76110b29e6010cef059dff17b: Status 404 returned error can't find the container with id 36f06475b258b116ecdb479aaaad602af29d9ba76110b29e6010cef059dff17b Jul 17 13:45:15.907823 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.907784 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v86dk" Jul 17 13:45:15.908909 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.908877 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012f0baa_3293_439e_b169_4400362a0857.slice/crio-d747a60491b83274e3ae37c3726f43791b4172dc99cc7ce8f7136d4357586b12 WatchSource:0}: Error finding container d747a60491b83274e3ae37c3726f43791b4172dc99cc7ce8f7136d4357586b12: Status 404 returned error can't find the container with id d747a60491b83274e3ae37c3726f43791b4172dc99cc7ce8f7136d4357586b12 Jul 17 13:45:15.916398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.916369 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rtgps" Jul 17 13:45:15.918167 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.918142 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3517e45a_1fe0_434f_b546_ef3db5198201.slice/crio-d373ab345c36b21402ec0f47d95e76e38fdc1a6da9222edc367b18b3f70879ac WatchSource:0}: Error finding container d373ab345c36b21402ec0f47d95e76e38fdc1a6da9222edc367b18b3f70879ac: Status 404 returned error can't find the container with id d373ab345c36b21402ec0f47d95e76e38fdc1a6da9222edc367b18b3f70879ac Jul 17 13:45:15.925151 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.925128 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:15.928567 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.928541 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181882a0_faee_4bd6_bce1_e4baf72269f8.slice/crio-70d13b585e50f742d05f4a96bca6956e36cb1affcdf0e061b3a3cdb8573dc0a9 WatchSource:0}: Error finding container 70d13b585e50f742d05f4a96bca6956e36cb1affcdf0e061b3a3cdb8573dc0a9: Status 404 returned error can't find the container with id 70d13b585e50f742d05f4a96bca6956e36cb1affcdf0e061b3a3cdb8573dc0a9 Jul 17 13:45:15.933829 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.933624 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" Jul 17 13:45:15.934915 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.934887 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcefb70_7a53_4e68_ad3f_6411e521bef8.slice/crio-894051bb3b67e071348ccad1d7ee6a329fe6da7155d3c02c8e8b38ae886df926 WatchSource:0}: Error finding container 894051bb3b67e071348ccad1d7ee6a329fe6da7155d3c02c8e8b38ae886df926: Status 404 returned error can't find the container with id 894051bb3b67e071348ccad1d7ee6a329fe6da7155d3c02c8e8b38ae886df926 Jul 17 13:45:15.943464 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.943430 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" Jul 17 13:45:15.945224 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.945194 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5055a98d_0e09_468e_a2f8_7cf602f1df79.slice/crio-d3ec124253e4b7a7ae710db6c3c7e475971f3eb4640f76206b1c51502f211094 WatchSource:0}: Error finding container d3ec124253e4b7a7ae710db6c3c7e475971f3eb4640f76206b1c51502f211094: Status 404 returned error can't find the container with id d3ec124253e4b7a7ae710db6c3c7e475971f3eb4640f76206b1c51502f211094 Jul 17 13:45:15.949344 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:15.949319 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bfpp" Jul 17 13:45:15.954211 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.954172 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a850987_4fc3_45b7_98a7_8fa8b483573a.slice/crio-fd75fa1b2a52d9eb3606c0dd9cd0791296be4c331fdfa393be407d0189e1169e WatchSource:0}: Error finding container fd75fa1b2a52d9eb3606c0dd9cd0791296be4c331fdfa393be407d0189e1169e: Status 404 returned error can't find the container with id fd75fa1b2a52d9eb3606c0dd9cd0791296be4c331fdfa393be407d0189e1169e Jul 17 13:45:15.960766 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:15.960742 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56b3af1_0414_43b6_a26f_1c8c7ab26463.slice/crio-166735ff83b2881546717ca048e27f356cfc2f672cb88bca02b312f4789aa125 WatchSource:0}: Error finding container 166735ff83b2881546717ca048e27f356cfc2f672cb88bca02b312f4789aa125: Status 404 returned error can't find the container with id 166735ff83b2881546717ca048e27f356cfc2f672cb88bca02b312f4789aa125 Jul 17 13:45:16.192547 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.192463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:16.192779 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.192605 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:16.192779 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.192697 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.192673638 +0000 UTC m=+3.953538633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:16.293460 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.293415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:16.293659 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.293545 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:16.293659 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.293627 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.293606667 +0000 UTC m=+4.054471659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:16.394269 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.394226 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:16.394516 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.394493 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:16.394570 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.394526 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:16.394570 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.394541 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:16.394698 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.394612 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.394591395 +0000 UTC m=+4.155456401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:16.651218 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.651117 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:40:15 +0000 UTC" deadline="2028-02-09 10:48:14.589423633 +0000 UTC" Jul 17 13:45:16.651218 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.651161 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13725h2m57.93826679s" Jul 17 13:45:16.673496 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.673450 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:16.673672 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:16.673625 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:16.683179 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.683143 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bfpp" event={"ID":"c56b3af1-0414-43b6-a26f-1c8c7ab26463","Type":"ContainerStarted","Data":"166735ff83b2881546717ca048e27f356cfc2f672cb88bca02b312f4789aa125"} Jul 17 13:45:16.684463 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.684432 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" event={"ID":"6a850987-4fc3-45b7-98a7-8fa8b483573a","Type":"ContainerStarted","Data":"fd75fa1b2a52d9eb3606c0dd9cd0791296be4c331fdfa393be407d0189e1169e"} Jul 17 13:45:16.685647 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.685608 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" event={"ID":"5055a98d-0e09-468e-a2f8-7cf602f1df79","Type":"ContainerStarted","Data":"d3ec124253e4b7a7ae710db6c3c7e475971f3eb4640f76206b1c51502f211094"} Jul 17 13:45:16.686815 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.686789 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2n5dq" event={"ID":"fbcefb70-7a53-4e68-ad3f-6411e521bef8","Type":"ContainerStarted","Data":"894051bb3b67e071348ccad1d7ee6a329fe6da7155d3c02c8e8b38ae886df926"} Jul 17 13:45:16.687959 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.687925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"b3bdd909b987d8f61312a166ba7134db36214aeffca2751e21d3a27327a5a9e2"} Jul 17 13:45:16.689115 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.689089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rtgps" event={"ID":"181882a0-faee-4bd6-bce1-e4baf72269f8","Type":"ContainerStarted","Data":"70d13b585e50f742d05f4a96bca6956e36cb1affcdf0e061b3a3cdb8573dc0a9"} Jul 17 13:45:16.690386 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.690345 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v86dk" event={"ID":"3517e45a-1fe0-434f-b546-ef3db5198201","Type":"ContainerStarted","Data":"d373ab345c36b21402ec0f47d95e76e38fdc1a6da9222edc367b18b3f70879ac"} Jul 17 13:45:16.691523 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.691502 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerStarted","Data":"d747a60491b83274e3ae37c3726f43791b4172dc99cc7ce8f7136d4357586b12"} Jul 17 13:45:16.692603 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:16.692585 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7vhz" event={"ID":"703ae377-e2ed-4493-957d-7fa6599ee17c","Type":"ContainerStarted","Data":"36f06475b258b116ecdb479aaaad602af29d9ba76110b29e6010cef059dff17b"} Jul 17 13:45:17.200961 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.200917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:17.201151 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.201116 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:17.201229 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.201200 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:19.201173791 +0000 UTC m=+5.962038784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:17.302019 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.301980 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:17.302178 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.302124 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:17.302239 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.302207 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:19.30218648 +0000 UTC m=+6.063051483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:17.403158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.403117 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:17.403336 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.403288 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:17.403336 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.403310 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:17.403336 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.403322 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:17.403499 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.403382 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:19.403364282 +0000 UTC m=+6.164229276 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:17.651828 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.651766 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:40:15 +0000 UTC" deadline="2028-05-03 08:32:26.522473566 +0000 UTC" Jul 17 13:45:17.651828 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.651824 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15738h47m8.870653842s" Jul 17 13:45:17.679890 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.677925 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:17.679890 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.678059 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:17.679890 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.678504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:17.679890 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:17.678620 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:17.705810 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.705225 2555 generic.go:356] "Generic (PLEG): container finished" podID="99ae320061f58d9bc26e8732459e276d" containerID="49881916fca40f64afadf6d630230cf0a6aeaa46031d5faad73f6e55cafcd0b9" exitCode=0 Jul 17 13:45:17.705810 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.705322 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"99ae320061f58d9bc26e8732459e276d","Type":"ContainerDied","Data":"49881916fca40f64afadf6d630230cf0a6aeaa46031d5faad73f6e55cafcd0b9"} Jul 17 13:45:17.712022 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:17.711988 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" event={"ID":"f128f886475b532dbbbd78614ba32f76","Type":"ContainerStarted","Data":"f7ae21ba051b74b94b9857fcdf230dc944cc677b62b5dc742d53b5d11e205ef1"} Jul 17 13:45:18.673132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:18.673099 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:18.673613 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:18.673238 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:18.730462 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:18.730352 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" event={"ID":"99ae320061f58d9bc26e8732459e276d","Type":"ContainerStarted","Data":"d35c6742d346e7487d17e5252ec952b669bd273a863c046b7ab2f1b26eee1b5d"} Jul 17 13:45:18.752400 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:18.751271 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-123.ec2.internal" podStartSLOduration=3.751246374 podStartE2EDuration="3.751246374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:17.753409206 +0000 UTC m=+4.514274223" watchObservedRunningTime="2026-07-17 13:45:18.751246374 +0000 UTC m=+5.512111391" Jul 17 13:45:19.220850 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:19.220181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:19.220850 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.220369 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:19.220850 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.220438 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:23.220419093 +0000 UTC m=+9.981284097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:19.322240 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:19.321534 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:19.322240 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.321722 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:19.322240 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.321809 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:23.321789541 +0000 UTC m=+10.082654539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:19.422664 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:19.421989 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:19.422664 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.422200 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:19.422664 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.422221 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:19.422664 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.422236 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:19.422664 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.422297 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:23.422278013 +0000 UTC m=+10.183143024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:19.673351 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:19.673271 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:19.673821 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.673403 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:19.674202 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:19.674033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:19.674202 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:19.674135 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:20.673615 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:20.673163 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:20.673615 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:20.673300 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:21.673210 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:21.673179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:21.673389 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:21.673191 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:21.673389 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:21.673314 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:21.673505 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:21.673402 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:22.673018 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:22.672981 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:22.673446 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:22.673122 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:23.252646 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:23.252607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:23.252871 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.252792 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:23.252871 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.252865 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:31.252845266 +0000 UTC m=+18.013710261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:23.353873 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:23.353718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:23.354069 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.353914 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:23.354069 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.353990 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:31.353970598 +0000 UTC m=+18.114835590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:23.454777 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:23.454652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:23.454957 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.454877 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:23.454957 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.454902 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:23.454957 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.454917 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:23.455077 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.454982 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:31.454964973 +0000 UTC m=+18.215829981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:23.674553 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:23.673896 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:23.674553 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.674005 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:23.674553 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:23.674369 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:23.674553 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:23.674471 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:24.672726 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:24.672690 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:24.672933 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:24.672840 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:25.673024 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:25.672989 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:25.673481 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:25.673124 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:25.673481 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:25.673176 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:25.673481 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:25.673296 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:26.673333 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:26.673119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:26.673333 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:26.673259 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:27.672705 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:27.672667 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:27.672705 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:27.672682 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:27.672950 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:27.672848 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:27.672950 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:27.672942 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:28.673403 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:28.673368 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:28.673820 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:28.673485 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:29.672851 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:29.672818 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:29.672851 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:29.672850 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:29.673060 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:29.672919 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:29.673060 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:29.673007 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:30.673456 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:30.673422 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:30.673980 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:30.673527 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:31.320408 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:31.320353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:31.320564 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.320517 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:31.320632 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.320596 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:45:47.320574511 +0000 UTC m=+34.081439512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:31.421203 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:31.421161 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:31.421390 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.421323 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:31.421453 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.421403 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:45:47.421380965 +0000 UTC m=+34.182245957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:31.522321 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:31.522283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:31.522509 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.522481 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:31.522509 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.522505 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:31.522612 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.522518 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:31.522612 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.522587 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:47.522567357 +0000 UTC m=+34.283432374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:31.673480 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:31.673395 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:31.673848 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.673498 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:31.673848 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:31.673560 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:31.673848 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:31.673681 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:32.672604 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:32.672560 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:32.672824 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:32.672696 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:33.673351 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:33.673316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:33.673800 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:33.673419 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:33.673800 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:33.673471 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:33.673800 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:33.673570 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:34.673411 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:34.673373 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:34.673833 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:34.673499 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:35.673825 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.673172 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:35.673825 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.673289 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:35.673825 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:35.673487 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:35.673825 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:35.673527 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:35.767811 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.767605 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2n5dq" event={"ID":"fbcefb70-7a53-4e68-ad3f-6411e521bef8","Type":"ContainerStarted","Data":"a750006ea62067bb6b3e6eb78ad4d9dea721d2fec0b3be22924eb6088c672680"} Jul 17 13:45:35.769246 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.769220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"9d0dcf1c764e96646c71a1c1cb7515799ba880ee010be6f24a04a5510d661658"} Jul 17 13:45:35.770700 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.770675 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rtgps" event={"ID":"181882a0-faee-4bd6-bce1-e4baf72269f8","Type":"ContainerStarted","Data":"4bc72d796b89d9a1809481dfc340f1531d2dfc5794ff12a214249d7347340c29"} Jul 17 13:45:35.772430 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.772399 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v86dk" event={"ID":"3517e45a-1fe0-434f-b546-ef3db5198201","Type":"ContainerStarted","Data":"428a45ab522fc5b182a0a043e7ee082b73e6808bdaa0059bf039fde47412dc9d"} Jul 17 13:45:35.774075 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.774047 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="742f9fc22cb93e78c174cfb0db93dae072795aaea2e7be460f517d557af9c963" exitCode=0 Jul 17 13:45:35.774193 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.774130 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"742f9fc22cb93e78c174cfb0db93dae072795aaea2e7be460f517d557af9c963"} Jul 17 13:45:35.775873 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.775831 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bfpp" event={"ID":"c56b3af1-0414-43b6-a26f-1c8c7ab26463","Type":"ContainerStarted","Data":"a785ff2d40a1fff39135bd917128c2f2dbd4c3a0fa413bd9d682a54918429e51"} Jul 17 13:45:35.777387 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.777367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" event={"ID":"6a850987-4fc3-45b7-98a7-8fa8b483573a","Type":"ContainerStarted","Data":"2639d485d31142d785371a9111478ea54e970c3a70c7003a5f75a6e7ec723659"} Jul 17 13:45:35.778840 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.778822 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" event={"ID":"5055a98d-0e09-468e-a2f8-7cf602f1df79","Type":"ContainerStarted","Data":"4833376401fe36469dedb14a3e88b998b532264795f0577879c998a8389c076b"} Jul 17 13:45:35.786042 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.786001 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2n5dq" podStartSLOduration=10.462634569 podStartE2EDuration="21.785988151s" totalImagesPullingTime="11.323353582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:35.785609136 +0000 UTC m=+22.546474150" watchObservedRunningTime="2026-07-17 13:45:35.785988151 +0000 UTC m=+22.546853164" Jul 17 13:45:35.786228 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.786209 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-123.ec2.internal" podStartSLOduration=20.786203788999998 podStartE2EDuration="20.786203789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:18.752363178 +0000 UTC m=+5.513228172" watchObservedRunningTime="2026-07-17 13:45:35.786203789 +0000 UTC m=+22.547068817" Jul 17 13:45:35.807131 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.807076 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lxqzv" podStartSLOduration=2.269846997 podStartE2EDuration="21.807056146s" totalImagesPullingTime="19.537209149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:35.807038317 +0000 UTC m=+22.567903325" watchObservedRunningTime="2026-07-17 13:45:35.807056146 +0000 UTC m=+22.567921160" Jul 17 13:45:35.827596 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.827534 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4bfpp" podStartSLOduration=10.527954672 podStartE2EDuration="21.827511225s" totalImagesPullingTime="11.299556553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:35.826899194 +0000 UTC m=+22.587764208" watchObservedRunningTime="2026-07-17 13:45:35.827511225 +0000 UTC m=+22.588376239" Jul 17 13:45:35.856952 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:35.856898 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v86dk" podStartSLOduration=2.226166354 podStartE2EDuration="21.856879816s" totalImagesPullingTime="19.630713462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:35.855289688 +0000 UTC m=+22.616154705" watchObservedRunningTime="2026-07-17 13:45:35.856879816 +0000 UTC m=+22.617744829" Jul 17 13:45:36.672904 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.672872 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:36.673066 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:36.672990 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:36.783331 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.783247 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7vhz" event={"ID":"703ae377-e2ed-4493-957d-7fa6599ee17c","Type":"ContainerStarted","Data":"37ca45c297d4939887677871621a7a5e5c099200c4e2ca34591705e2c2c7305c"} Jul 17 13:45:36.788771 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.788686 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"9cf82595f5097f90f5e0b9205170540970e089c479cb64499ba6bfb411dd50b1"} Jul 17 13:45:36.788948 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.788781 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"5b394576dee4437a5af3e874de69a3dd963ea0ee5767e50322de8da7d39826d3"} Jul 17 13:45:36.788948 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.788827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"48ece7b3f95e754e17361aa1d2ce1fe7c74428c1b9463f9e5e357d4bfe652683"} Jul 17 13:45:36.788948 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.788868 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"542aecb77dd7f3a85b074391e3189b5e80b23e00c8527e6ae2a6315069b2b772"} Jul 17 13:45:36.788948 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.788911 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"303bbea178537a6a4afab687c027317c917230f0e152f877b505a3780f405ab8"} Jul 17 13:45:36.799810 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.799736 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rtgps" podStartSLOduration=11.467814421 podStartE2EDuration="22.799714601s" totalImagesPullingTime="11.33190018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:35.923596339 +0000 UTC m=+22.684461353" watchObservedRunningTime="2026-07-17 13:45:36.799714601 +0000 UTC m=+23.560579616" Jul 17 13:45:36.800146 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:36.800117 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t7vhz" podStartSLOduration=11.438732572 podStartE2EDuration="22.800108794s" totalImagesPullingTime="11.361376222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:36.799436665 +0000 UTC m=+23.560301735" watchObservedRunningTime="2026-07-17 13:45:36.800108794 +0000 UTC m=+23.560973808" Jul 17 13:45:37.411991 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.411960 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:37.412609 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.412585 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:37.503445 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.503427 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 13:45:37.639886 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.639708 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T13:45:37.503441501Z","UUID":"e0197c9d-ab30-4efc-a509-1d4f2f4ef6b1","Handler":null,"Name":"","Endpoint":""} Jul 17 13:45:37.641441 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.641417 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 13:45:37.641565 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.641456 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 13:45:37.676157 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.676126 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:37.676337 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:37.676238 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:37.676337 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.676296 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:37.676451 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:37.676404 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:37.792353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.792313 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" event={"ID":"5055a98d-0e09-468e-a2f8-7cf602f1df79","Type":"ContainerStarted","Data":"9f0e43b2f6bf00882735972c82bf894359f3696a2383a14d5ab1dbb5e79d452a"} Jul 17 13:45:37.792849 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.792622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:37.793139 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:37.793124 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2n5dq" Jul 17 13:45:38.672598 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:38.672508 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:38.672772 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:38.672627 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:38.798411 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:38.798376 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"c7cb8e6b2c6e0259a9f81204fecb18b6959af1fd898cc81e174006c5ad19b8df"} Jul 17 13:45:39.673807 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:39.673598 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:39.673994 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:39.673645 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:39.673994 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:39.673912 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:39.673994 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:39.673960 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:39.802931 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:39.802892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" event={"ID":"5055a98d-0e09-468e-a2f8-7cf602f1df79","Type":"ContainerStarted","Data":"de5605b15bf901e74c99238479e24cc9e8bdac307f871715291d4a8cc61796dc"} Jul 17 13:45:39.831653 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:39.831597 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghfv6" podStartSLOduration=3.202192574 podStartE2EDuration="25.831576504s" totalImagesPullingTime="22.62938393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:39.830821793 +0000 UTC m=+26.591686807" watchObservedRunningTime="2026-07-17 13:45:39.831576504 +0000 UTC m=+26.592441519" Jul 17 13:45:40.672867 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:40.672820 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:40.673050 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:40.672948 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:41.672970 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.672789 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:41.673655 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.672789 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:41.673655 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:41.673064 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:41.673655 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:41.673143 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:41.808871 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.808834 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="f9609256e8dc1559eb58f2f48545ddf26a9ccb414d0ed14ecadaa3d6ac86cee3" exitCode=0 Jul 17 13:45:41.809059 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.808919 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"f9609256e8dc1559eb58f2f48545ddf26a9ccb414d0ed14ecadaa3d6ac86cee3"} Jul 17 13:45:41.812628 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.812606 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" event={"ID":"6f8757ff-c282-4729-b550-202d7a617d91","Type":"ContainerStarted","Data":"c0b19d2f7a005e5790485f0093f5f7645d3675f0763623fe302845f828d02b88"} Jul 17 13:45:41.812984 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.812953 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:41.828251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:41.828224 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:42.673118 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.673081 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:42.673639 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:42.673219 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:42.814935 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.814911 2555 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 17 13:45:42.815347 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.815328 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:42.829978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.829951 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:42.853573 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.853544 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:45:42.869844 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.869799 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" podStartSLOduration=10.202674981 podStartE2EDuration="29.869783177s" totalImagesPullingTime="19.667108196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:41.892613726 +0000 UTC m=+28.653478743" watchObservedRunningTime="2026-07-17 13:45:42.869783177 +0000 UTC m=+29.630648190" Jul 17 13:45:42.925658 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.925581 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r9xs2"] Jul 17 13:45:42.925829 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.925698 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:42.925829 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:42.925802 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:42.932426 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.932389 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v2nz4"] Jul 17 13:45:42.932561 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.932551 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:42.932666 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:42.932648 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:42.936318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.936296 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8kgqr"] Jul 17 13:45:42.936434 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:42.936399 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:42.936505 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:42.936484 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:43.818778 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:43.818735 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="343e2083c3b55ea436e0e90fd1ee380475b9ee6e3554fa817da0484c05137666" exitCode=0 Jul 17 13:45:43.819183 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:43.818821 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"343e2083c3b55ea436e0e90fd1ee380475b9ee6e3554fa817da0484c05137666"} Jul 17 13:45:44.672730 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:44.672605 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:44.672730 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:44.672640 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:44.672730 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:44.672643 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:44.672926 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:44.672742 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:44.672926 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:44.672858 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:44.673006 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:44.672955 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:45.826044 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:45.825859 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="8ee01e4688e279fddee791cfe97dd5ca3943f82cb956ab41e1b3015dfa481bc7" exitCode=0 Jul 17 13:45:45.826044 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:45.825930 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"8ee01e4688e279fddee791cfe97dd5ca3943f82cb956ab41e1b3015dfa481bc7"} Jul 17 13:45:46.672845 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:46.672806 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:46.672845 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:46.672831 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:46.672845 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:46.672847 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:46.673092 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:46.672948 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r9xs2" podUID="58a9967a-28c3-45cd-af44-13d2619266fc" Jul 17 13:45:46.673092 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:46.673062 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8kgqr" podUID="707ee971-f0b2-4001-ba7f-dfad9dd4a140" Jul 17 13:45:46.673162 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:46.673147 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:45:47.337737 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:47.337694 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:47.338191 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.337855 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:47.338191 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.337923 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.33790774 +0000 UTC m=+66.098772732 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:45:47.438694 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:47.438657 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:47.438912 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.438844 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:47.438979 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.438927 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret podName:58a9967a-28c3-45cd-af44-13d2619266fc nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.438907481 +0000 UTC m=+66.199772472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret") pod "global-pull-secret-syncer-r9xs2" (UID: "58a9967a-28c3-45cd-af44-13d2619266fc") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:45:47.539445 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:47.539409 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:47.539626 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.539583 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:45:47.539626 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.539606 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:45:47.539626 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.539618 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jbm99 for pod openshift-network-diagnostics/network-check-target-8kgqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:47.539741 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:47.539676 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99 podName:707ee971-f0b2-4001-ba7f-dfad9dd4a140 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.539662261 +0000 UTC m=+66.300527254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbm99" (UniqueName: "kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99") pod "network-check-target-8kgqr" (UID: "707ee971-f0b2-4001-ba7f-dfad9dd4a140") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:45:48.056986 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.056961 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-123.ec2.internal" event="NodeReady" Jul 17 13:45:48.057147 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.057080 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 13:45:48.104976 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.104940 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst"] Jul 17 13:45:48.107080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.107054 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:45:48.107219 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.107182 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.109475 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.109456 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887"] Jul 17 13:45:48.109598 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.109580 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.112023 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.112001 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g"] Jul 17 13:45:48.112125 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.112116 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.114821 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.114801 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n"] Jul 17 13:45:48.114941 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.114922 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.117545 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.117446 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.118128 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.117987 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 17 13:45:48.118128 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.117999 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 13:45:48.118257 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118226 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118573 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118625 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118693 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118731 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118812 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118820 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 17 13:45:48.118927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.118656 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 17 13:45:48.120041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120023 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tpx5k\"" Jul 17 13:45:48.120449 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120348 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cnxqs\"" Jul 17 13:45:48.120449 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120371 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bwc6l\"" Jul 17 13:45:48.120449 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.120372 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-133-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" Jul 17 13:45:48.120449 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120388 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 17 13:45:48.120706 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120483 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 17 13:45:48.120706 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.120529 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 17 13:45:48.126985 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.126961 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 13:45:48.131895 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.131870 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst"] Jul 17 13:45:48.136935 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.136818 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:45:48.141453 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.141430 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n"] Jul 17 13:45:48.149100 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.149048 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887"] Jul 17 13:45:48.150070 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.150027 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rq7pc"] Jul 17 13:45:48.153136 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.153082 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vfrhc"] Jul 17 13:45:48.153256 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.153237 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.155882 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.155857 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.156858 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.156828 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 13:45:48.156858 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.156854 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 13:45:48.157000 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.156866 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v8874\"" Jul 17 13:45:48.157000 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.156961 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 13:45:48.159451 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.159431 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 13:45:48.159702 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.159682 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 13:45:48.159926 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.159805 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gl8vw\"" Jul 17 13:45:48.163178 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.163156 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g"] Jul 17 13:45:48.170448 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.170423 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rq7pc"] Jul 17 13:45:48.171464 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.171443 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vfrhc"] Jul 17 13:45:48.244800 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.244766 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlkl2\" (UniqueName: \"kubernetes.io/projected/7fa85d62-bc6d-437b-a286-ed8ad35ae662-kube-api-access-nlkl2\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.244982 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.244817 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa85d62-bc6d-437b-a286-ed8ad35ae662-config-volume\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.244982 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.244881 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.244982 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.244941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.244982 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.244969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245034 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ce42e4-32eb-41ad-af5f-a309b312e472-additional-service-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245063 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245084 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245111 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ltv\" (UniqueName: \"kubernetes.io/projected/d2ce42e4-32eb-41ad-af5f-a309b312e472-kube-api-access-v4ltv\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245140 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245189 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa85d62-bc6d-437b-a286-ed8ad35ae662-tmp-dir\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245203 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f45038c-8853-4be2-8d2e-368eddeb996d-klusterlet-config\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245220 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245250 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245283 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245306 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4wm\" (UniqueName: \"kubernetes.io/projected/6f45038c-8853-4be2-8d2e-368eddeb996d-kube-api-access-xz4wm\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245344 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01d2f7b7-1f92-4d12-99f1-f1524f347729-cert\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.245418 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245375 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245436 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7dw\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245519 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245543 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f45038c-8853-4be2-8d2e-368eddeb996d-tmp\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245587 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47q62\" (UniqueName: \"kubernetes.io/projected/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-kube-api-access-47q62\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245616 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jcsp\" (UniqueName: \"kubernetes.io/projected/01d2f7b7-1f92-4d12-99f1-f1524f347729-kube-api-access-5jcsp\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.245903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.245706 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.346429 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa85d62-bc6d-437b-a286-ed8ad35ae662-tmp-dir\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.346429 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346429 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f45038c-8853-4be2-8d2e-368eddeb996d-klusterlet-config\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346454 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346506 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4wm\" (UniqueName: \"kubernetes.io/projected/6f45038c-8853-4be2-8d2e-368eddeb996d-kube-api-access-xz4wm\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.346620 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.346626 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346646 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01d2f7b7-1f92-4d12-99f1-f1524f347729-cert\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.346691 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.846673031 +0000 UTC m=+35.607538028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7dw\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346803 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f45038c-8853-4be2-8d2e-368eddeb996d-tmp\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.346852 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.846833697 +0000 UTC m=+35.607698688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:45:48.346973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346893 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47q62\" (UniqueName: \"kubernetes.io/projected/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-kube-api-access-47q62\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346925 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jcsp\" (UniqueName: \"kubernetes.io/projected/01d2f7b7-1f92-4d12-99f1-f1524f347729-kube-api-access-5jcsp\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.346893 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fa85d62-bc6d-437b-a286-ed8ad35ae662-tmp-dir\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347108 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347150 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f45038c-8853-4be2-8d2e-368eddeb996d-tmp\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347199 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlkl2\" (UniqueName: \"kubernetes.io/projected/7fa85d62-bc6d-437b-a286-ed8ad35ae662-kube-api-access-nlkl2\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347245 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa85d62-bc6d-437b-a286-ed8ad35ae662-config-volume\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347275 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ce42e4-32eb-41ad-af5f-a309b312e472-additional-service-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347371 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347379 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.347701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.347411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.348398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.348141 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.348398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.348358 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ce42e4-32eb-41ad-af5f-a309b312e472-additional-service-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.349354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa85d62-bc6d-437b-a286-ed8ad35ae662-config-volume\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.349354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349086 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.349354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ltv\" (UniqueName: \"kubernetes.io/projected/d2ce42e4-32eb-41ad-af5f-a309b312e472-kube-api-access-v4ltv\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.349354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349230 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.349354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.349762 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.349727 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.350502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.350481 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:45:48.350591 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.350504 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:45:48.350591 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.350561 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.850544716 +0000 UTC m=+35.611409707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:45:48.352316 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352103 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01d2f7b7-1f92-4d12-99f1-f1524f347729-cert\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.352316 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352123 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.352316 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352130 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.352316 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352202 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-ca\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.352316 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352211 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f45038c-8853-4be2-8d2e-368eddeb996d-klusterlet-config\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.352716 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.352842 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.352823 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.353105 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.353083 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d2ce42e4-32eb-41ad-af5f-a309b312e472-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.354105 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.354084 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.357529 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.357505 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4wm\" (UniqueName: \"kubernetes.io/projected/6f45038c-8853-4be2-8d2e-368eddeb996d-kube-api-access-xz4wm\") pod \"klusterlet-addon-workmgr-86f86f6987-ln57n\" (UID: \"6f45038c-8853-4be2-8d2e-368eddeb996d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:48.357804 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.357786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.368806 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.368541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ltv\" (UniqueName: \"kubernetes.io/projected/d2ce42e4-32eb-41ad-af5f-a309b312e472-kube-api-access-v4ltv\") pod \"cluster-proxy-proxy-agent-859c669d89-cf887\" (UID: \"d2ce42e4-32eb-41ad-af5f-a309b312e472\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:48.368965 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.368920 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jcsp\" (UniqueName: \"kubernetes.io/projected/01d2f7b7-1f92-4d12-99f1-f1524f347729-kube-api-access-5jcsp\") pod \"ingress-canary-rq7pc\" (UID: \"01d2f7b7-1f92-4d12-99f1-f1524f347729\") " pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.369108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.369089 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlkl2\" (UniqueName: \"kubernetes.io/projected/7fa85d62-bc6d-437b-a286-ed8ad35ae662-kube-api-access-nlkl2\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.370685 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.370661 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7dw\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.371259 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.371237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47q62\" (UniqueName: \"kubernetes.io/projected/aeb65c5d-3792-4cd3-ae34-9403c7c2786d-kube-api-access-47q62\") pod \"managed-serviceaccount-addon-agent-67666f755b-6ssst\" (UID: \"aeb65c5d-3792-4cd3-ae34-9403c7c2786d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:48.410744 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.410707 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:48.473072 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.473040 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rq7pc" Jul 17 13:45:48.545177 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.545121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:48.639421 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.639312 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rq7pc"] Jul 17 13:45:48.642836 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:48.642804 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d2f7b7_1f92_4d12_99f1_f1524f347729.slice/crio-0ed10e5e458828b86d1bbdc0963d4c09b73d5e80e72f9db880862ba45d186b8b WatchSource:0}: Error finding container 0ed10e5e458828b86d1bbdc0963d4c09b73d5e80e72f9db880862ba45d186b8b: Status 404 returned error can't find the container with id 0ed10e5e458828b86d1bbdc0963d4c09b73d5e80e72f9db880862ba45d186b8b Jul 17 13:45:48.673502 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.673461 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:45:48.673502 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.673484 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:45:48.673502 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.673504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:45:48.682632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.682467 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 13:45:48.682879 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.682855 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 13:45:48.682976 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.682944 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 13:45:48.683340 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.683316 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnsb\"" Jul 17 13:45:48.684172 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.684153 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 13:45:48.684269 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.684227 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tpk25\"" Jul 17 13:45:48.835396 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.835284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rq7pc" event={"ID":"01d2f7b7-1f92-4d12-99f1-f1524f347729","Type":"ContainerStarted","Data":"0ed10e5e458828b86d1bbdc0963d4c09b73d5e80e72f9db880862ba45d186b8b"} Jul 17 13:45:48.854294 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.854255 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:48.854468 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.854313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:48.854468 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:48.854361 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:48.854468 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854428 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:45:48.854468 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854449 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:45:48.854660 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854478 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:45:48.854660 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854490 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:45:48.854660 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854544 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:49.85452357 +0000 UTC m=+36.615388567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:45:48.854660 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854563 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:49.854553961 +0000 UTC m=+36.615418959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:45:48.854660 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:48.854578 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:49.854570216 +0000 UTC m=+36.615435210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:45:49.428573 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.428534 2555 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 17 13:45:49.429227 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.428623 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:45:49.443123 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.443091 2555 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 17 13:45:49.443279 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.443167 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:45:49.464187 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.464151 2555 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 17 13:45:49.464411 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.464238 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:49.598566 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.598531 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 17 13:45:49.864576 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.864543 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:49.864813 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.864608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:49.864813 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:49.864648 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:49.864813 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864711 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:45:49.864813 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864725 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:45:49.865074 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864815 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:45:49.865074 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864827 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:51.864803705 +0000 UTC m=+38.625668716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:45:49.865074 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864733 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:45:49.865074 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864868 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:51.864852008 +0000 UTC m=+38.625717002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:45:49.865074 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:49.864886 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:51.864877239 +0000 UTC m=+38.625742229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:45:50.646900 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.646875 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n"] Jul 17 13:45:50.648600 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:50.648571 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f45038c_8853_4be2_8d2e_368eddeb996d.slice/crio-7142a33831f740b62cffd79d1bb63f6df502d50428d0f0a55df31f293c81f451 WatchSource:0}: Error finding container 7142a33831f740b62cffd79d1bb63f6df502d50428d0f0a55df31f293c81f451: Status 404 returned error can't find the container with id 7142a33831f740b62cffd79d1bb63f6df502d50428d0f0a55df31f293c81f451 Jul 17 13:45:50.659235 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.659210 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887"] Jul 17 13:45:50.661702 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.661680 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst"] Jul 17 13:45:50.664409 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:50.664378 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2ce42e4_32eb_41ad_af5f_a309b312e472.slice/crio-6b573ef51873a53f36376d116a6db5f20f7641690eda38891f0c0315764461dc WatchSource:0}: Error finding container 6b573ef51873a53f36376d116a6db5f20f7641690eda38891f0c0315764461dc: Status 404 returned error can't find the container with id 6b573ef51873a53f36376d116a6db5f20f7641690eda38891f0c0315764461dc Jul 17 13:45:50.665677 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:45:50.665654 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeb65c5d_3792_4cd3_ae34_9403c7c2786d.slice/crio-d4273abc024ba7721bfcc49a3fff7d883637b03ec606f6a5a5c62d88c51aaa17 WatchSource:0}: Error finding container d4273abc024ba7721bfcc49a3fff7d883637b03ec606f6a5a5c62d88c51aaa17: Status 404 returned error can't find the container with id d4273abc024ba7721bfcc49a3fff7d883637b03ec606f6a5a5c62d88c51aaa17 Jul 17 13:45:50.841265 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.841233 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerStarted","Data":"7142a33831f740b62cffd79d1bb63f6df502d50428d0f0a55df31f293c81f451"} Jul 17 13:45:50.843892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.843865 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="35eeede84e8a34b6861dd881311f1acf329bc26e85cd8a34ed2dcfeeb57907a0" exitCode=0 Jul 17 13:45:50.844023 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.843935 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"35eeede84e8a34b6861dd881311f1acf329bc26e85cd8a34ed2dcfeeb57907a0"} Jul 17 13:45:50.845046 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.845011 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerStarted","Data":"d4273abc024ba7721bfcc49a3fff7d883637b03ec606f6a5a5c62d88c51aaa17"} Jul 17 13:45:50.846044 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:50.846022 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerStarted","Data":"6b573ef51873a53f36376d116a6db5f20f7641690eda38891f0c0315764461dc"} Jul 17 13:45:51.854826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:51.854512 2555 generic.go:356] "Generic (PLEG): container finished" podID="012f0baa-3293-439e-b169-4400362a0857" containerID="2e6612d39e497a1b463c6cdaa562176c50a3f85fd3287f0aa41148ef9b203e9c" exitCode=0 Jul 17 13:45:51.854826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:51.854802 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerDied","Data":"2e6612d39e497a1b463c6cdaa562176c50a3f85fd3287f0aa41148ef9b203e9c"} Jul 17 13:45:51.884551 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:51.884512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:51.884686 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:51.884582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:51.884743 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:51.884703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:51.884854 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.884786 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:45:51.884911 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.884892 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:45:51.884911 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.884906 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:55.884885225 +0000 UTC m=+42.645750220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:45:51.885009 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.884915 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:45:51.885009 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.884980 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:55.884961454 +0000 UTC m=+42.645826446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:45:51.885470 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.885448 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:45:51.885558 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:51.885551 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:55.885532156 +0000 UTC m=+42.646397150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:45:52.869887 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:52.869837 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbpph" event={"ID":"012f0baa-3293-439e-b169-4400362a0857","Type":"ContainerStarted","Data":"22a542c7b7eed285db0ae458bf36fb415a4b179e826e5b3098313c00114fc01b"} Jul 17 13:45:53.035577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:53.035542 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:53.197170 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:53.197090 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:54.283764 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:54.283703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:54.466186 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:54.466139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:54.538215 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:54.538126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:55.921598 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:55.921558 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:45:55.921598 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:55.921599 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:45:55.922109 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:55.921676 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:45:55.922109 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.921771 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:45:55.922109 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.921835 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:45:55.922258 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.922133 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:03.922098526 +0000 UTC m=+50.682963517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:45:55.922258 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.922200 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:45:55.922258 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.922246 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:03.92221615 +0000 UTC m=+50.683081158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:45:55.922402 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.922330 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:45:55.922618 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:45:55.922604 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:03.92257362 +0000 UTC m=+50.683438630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:45:58.404243 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.404205 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:58.474516 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.474481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:58.887282 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.887252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rq7pc" event={"ID":"01d2f7b7-1f92-4d12-99f1-f1524f347729","Type":"ContainerStarted","Data":"03136e0b2970c0b5c3160b0449e3a325e6bd2e9389c8ab87c084dc950f979067"} Jul 17 13:45:58.888667 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.888641 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerStarted","Data":"77f558c8d55445443fb9081e050ca8fbe91b1966c2f7264aa7317c6e18031c3f"} Jul 17 13:45:58.888802 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.888788 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:58.890353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.890329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerStarted","Data":"3448b1dd95bc3bce5a3d27c1754a9f2600c3209af38acce9bd24a0036ea879d0"} Jul 17 13:45:58.890692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.890666 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:45:58.892216 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.892198 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerStarted","Data":"d3802d02e4de4d811329da8f29efd668b09cf5e292dabe2c1edcb90d0710f0a1"} Jul 17 13:45:58.892298 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.892225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerStarted","Data":"14bd7ff26bf1682248136448374033670f1c8839afd9db6516592fd3af407d64"} Jul 17 13:45:58.908898 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.908850 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xbpph" podStartSLOduration=21.20960548 podStartE2EDuration="44.908834666s" totalImagesPullingTime="23.699229186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:52.898819802 +0000 UTC m=+39.659684817" watchObservedRunningTime="2026-07-17 13:45:58.908834666 +0000 UTC m=+45.669699678" Jul 17 13:45:58.909420 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.909390 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rq7pc" podStartSLOduration=0.969800331 podStartE2EDuration="10.909380721s" totalImagesPullingTime="9.93958039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:58.907377462 +0000 UTC m=+45.668242476" watchObservedRunningTime="2026-07-17 13:45:58.909380721 +0000 UTC m=+45.670245733" Jul 17 13:45:58.925870 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.925809 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" podStartSLOduration=16.010076669 podStartE2EDuration="23.925787297s" totalImagesPullingTime="7.915710628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:58.925606649 +0000 UTC m=+45.686471662" watchObservedRunningTime="2026-07-17 13:45:58.925787297 +0000 UTC m=+45.686652312" Jul 17 13:45:58.952290 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:58.951949 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" podStartSLOduration=16.001818316 podStartE2EDuration="23.951928013s" totalImagesPullingTime="7.950109697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:58.951002496 +0000 UTC m=+45.711867509" watchObservedRunningTime="2026-07-17 13:45:58.951928013 +0000 UTC m=+45.712793021" Jul 17 13:45:59.897367 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:59.897330 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerStarted","Data":"4c4357d933fc0467eb1cb38b890fe70556cb4abdeda99aaba84d42bb416b7275"} Jul 17 13:45:59.919068 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:45:59.919013 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" podStartSLOduration=16.999840529 podStartE2EDuration="24.918994011s" totalImagesPullingTime="7.919153482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:59.91829323 +0000 UTC m=+46.679158244" watchObservedRunningTime="2026-07-17 13:45:59.918994011 +0000 UTC m=+46.679859027" Jul 17 13:46:00.784628 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:00.784587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:02.328052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:02.328016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:02.424972 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:02.424942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:03.990652 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:03.990590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:46:03.990652 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:03.990662 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:03.990725 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990772 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990849 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990859 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990868 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990856 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.990839774 +0000 UTC m=+66.751704764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990964 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.990948329 +0000 UTC m=+66.751813323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:46:03.991170 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:03.990987 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:19.990976805 +0000 UTC m=+66.751841798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:46:14.837558 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:14.837530 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltngj" Jul 17 13:46:15.127542 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:15.127445 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:15.228179 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:15.228140 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:16.139069 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:16.139029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:16.198209 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:16.198172 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:19.423191 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.423145 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:46:19.426702 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.426683 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 13:46:19.434341 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:19.434312 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 17 13:46:19.434444 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:19.434390 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:47:23.434365856 +0000 UTC m=+130.195230847 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : secret "metrics-daemon-secret" not found Jul 17 13:46:19.524056 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.524006 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:46:19.526731 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.526713 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 13:46:19.538305 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.538283 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58a9967a-28c3-45cd-af44-13d2619266fc-original-pull-secret\") pod \"global-pull-secret-syncer-r9xs2\" (UID: \"58a9967a-28c3-45cd-af44-13d2619266fc\") " pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:46:19.602984 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.602931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r9xs2" Jul 17 13:46:19.624955 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.624926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:46:19.627874 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.627856 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 13:46:19.638544 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.638517 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 13:46:19.649610 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.649538 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbm99\" (UniqueName: \"kubernetes.io/projected/707ee971-f0b2-4001-ba7f-dfad9dd4a140-kube-api-access-jbm99\") pod \"network-check-target-8kgqr\" (UID: \"707ee971-f0b2-4001-ba7f-dfad9dd4a140\") " pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:46:19.730451 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.730418 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r9xs2"] Jul 17 13:46:19.734348 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:46:19.734313 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a9967a_28c3_45cd_af44_13d2619266fc.slice/crio-c6f811a04a775c8b43496f075494f5e6d132a42f59f630dd485aead9755ff9d3 WatchSource:0}: Error finding container c6f811a04a775c8b43496f075494f5e6d132a42f59f630dd485aead9755ff9d3: Status 404 returned error can't find the container with id c6f811a04a775c8b43496f075494f5e6d132a42f59f630dd485aead9755ff9d3 Jul 17 13:46:19.899654 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.899623 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tpk25\"" Jul 17 13:46:19.907555 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.907536 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:46:19.959577 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:19.959548 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r9xs2" event={"ID":"58a9967a-28c3-45cd-af44-13d2619266fc","Type":"ContainerStarted","Data":"c6f811a04a775c8b43496f075494f5e6d132a42f59f630dd485aead9755ff9d3"} Jul 17 13:46:20.027365 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.027333 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:46:20.027534 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.027406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:46:20.027534 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.027436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:46:20.027534 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027490 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:46:20.027649 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027550 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:46:20.027649 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027566 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:46:20.027649 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027566 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:46:20.027649 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027571 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:52.027553251 +0000 UTC m=+98.788418243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:46:20.027846 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027656 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:52.02763599 +0000 UTC m=+98.788500994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:46:20.027846 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:20.027685 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:52.027678107 +0000 UTC m=+98.788543099 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:46:20.029850 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.029823 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8kgqr"] Jul 17 13:46:20.033008 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:46:20.032985 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707ee971_f0b2_4001_ba7f_dfad9dd4a140.slice/crio-05291ef6e3cb35428f61afea47dcc9b478b13f6b5e9e07b721194e9e9c957e1c WatchSource:0}: Error finding container 05291ef6e3cb35428f61afea47dcc9b478b13f6b5e9e07b721194e9e9c957e1c: Status 404 returned error can't find the container with id 05291ef6e3cb35428f61afea47dcc9b478b13f6b5e9e07b721194e9e9c957e1c Jul 17 13:46:20.377354 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.377315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:20.437395 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.437363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:20.964993 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:20.964956 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8kgqr" event={"ID":"707ee971-f0b2-4001-ba7f-dfad9dd4a140","Type":"ContainerStarted","Data":"05291ef6e3cb35428f61afea47dcc9b478b13f6b5e9e07b721194e9e9c957e1c"} Jul 17 13:46:25.982817 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:25.982782 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8kgqr" event={"ID":"707ee971-f0b2-4001-ba7f-dfad9dd4a140","Type":"ContainerStarted","Data":"4e4bb4d59bd5ea9aa9005bdce903cfe6349ff0ca930b79c9fef0051ac3a5c4cf"} Jul 17 13:46:25.983318 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:25.982886 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:46:25.984155 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:25.984133 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r9xs2" event={"ID":"58a9967a-28c3-45cd-af44-13d2619266fc","Type":"ContainerStarted","Data":"907dbbb893910164d53439e69e20b8a7f5d1a5738c4cc59f4de3773f900ca795"} Jul 17 13:46:26.000301 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:26.000254 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8kgqr" podStartSLOduration=67.125330007 podStartE2EDuration="1m12.000239812s" totalImagesPullingTime="4.874909805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:46:25.99962133 +0000 UTC m=+72.760486345" watchObservedRunningTime="2026-07-17 13:46:26.000239812 +0000 UTC m=+72.761104824" Jul 17 13:46:26.014691 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:26.014640 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r9xs2" podStartSLOduration=66.836385295 podStartE2EDuration="1m12.014623258s" totalImagesPullingTime="5.178237963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:46:26.013922681 +0000 UTC m=+72.774787716" watchObservedRunningTime="2026-07-17 13:46:26.014623258 +0000 UTC m=+72.775488271" Jul 17 13:46:28.435914 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:28.435876 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:28.492889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:28.492850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:52.072937 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:52.072798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:46:52.072937 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:52.072849 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:46:52.072937 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:52.072924 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.072948 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073009 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073022 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073037 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073015 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:47:56.072998905 +0000 UTC m=+162.833863900 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073086 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:47:56.073068552 +0000 UTC m=+162.833933543 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:46:52.073502 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:46:52.073103 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:47:56.073095766 +0000 UTC m=+162.833960757 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:46:56.989876 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:56.989846 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8kgqr" Jul 17 13:46:57.380206 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:57.380113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:46:57.434298 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:46:57.434260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:23.506985 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:23.506941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:47:23.507456 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:23.507088 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 17 13:47:23.507456 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:23.507157 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:49:25.507138866 +0000 UTC m=+252.268003858 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : secret "metrics-daemon-secret" not found Jul 17 13:47:51.151798 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:51.151722 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" Jul 17 13:47:51.157900 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:51.157867 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" podUID="ac6513eb-e134-4479-a0c5-2ec3378bc7b1" Jul 17 13:47:51.179708 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:51.179675 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vfrhc" podUID="7fa85d62-bc6d-437b-a286-ed8ad35ae662" Jul 17 13:47:51.236873 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:51.236844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:47:51.237037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:51.236844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:47:51.237037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:51.236844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:47:51.689011 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:51.688962 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:47:56.154683 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:56.154640 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:47:56.154683 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:56.154690 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:56.154714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154835 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154859 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fdc484b8d-h4ght: secret "image-registry-tls" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154918 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls podName:6037aea9-7003-4e2d-a869-496dc44409a8 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:58.154902101 +0000 UTC m=+284.915767092 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls") pod "image-registry-fdc484b8d-h4ght" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8") : secret "image-registry-tls" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154837 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154977 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls podName:7fa85d62-bc6d-437b-a286-ed8ad35ae662 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:58.154965241 +0000 UTC m=+284.915830231 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls") pod "dns-default-vfrhc" (UID: "7fa85d62-bc6d-437b-a286-ed8ad35ae662") : secret "dns-default-metrics-tls" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.154843 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 13:47:56.155209 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:47:56.155011 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert podName:ac6513eb-e134-4479-a0c5-2ec3378bc7b1 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:58.155004271 +0000 UTC m=+284.915869261 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-j2q6g" (UID: "ac6513eb-e134-4479-a0c5-2ec3378bc7b1") : secret "networking-console-plugin-cert" not found Jul 17 13:47:58.889963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:58.889914 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" podUID="6f45038c-8853-4be2-8d2e-368eddeb996d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jul 17 13:47:59.259697 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.259668 2555 generic.go:356] "Generic (PLEG): container finished" podID="6f45038c-8853-4be2-8d2e-368eddeb996d" containerID="77f558c8d55445443fb9081e050ca8fbe91b1966c2f7264aa7317c6e18031c3f" exitCode=1 Jul 17 13:47:59.259875 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.259746 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerDied","Data":"77f558c8d55445443fb9081e050ca8fbe91b1966c2f7264aa7317c6e18031c3f"} Jul 17 13:47:59.260094 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.260079 2555 scope.go:117] "RemoveContainer" containerID="77f558c8d55445443fb9081e050ca8fbe91b1966c2f7264aa7317c6e18031c3f" Jul 17 13:47:59.261271 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.261248 2555 generic.go:356] "Generic (PLEG): container finished" podID="aeb65c5d-3792-4cd3-ae34-9403c7c2786d" containerID="3448b1dd95bc3bce5a3d27c1754a9f2600c3209af38acce9bd24a0036ea879d0" exitCode=255 Jul 17 13:47:59.261349 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.261301 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerDied","Data":"3448b1dd95bc3bce5a3d27c1754a9f2600c3209af38acce9bd24a0036ea879d0"} Jul 17 13:47:59.261562 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.261545 2555 scope.go:117] "RemoveContainer" containerID="3448b1dd95bc3bce5a3d27c1754a9f2600c3209af38acce9bd24a0036ea879d0" Jul 17 13:47:59.429856 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.429799 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:47:59.465291 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:47:59.465243 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:48:00.265558 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:48:00.265524 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerStarted","Data":"41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61"} Jul 17 13:48:00.266041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:48:00.265701 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:48:00.267134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:48:00.267091 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:48:00.267868 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:48:00.267843 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerStarted","Data":"1296b5c748ff653d99ef26d43a2116f445b8983626a937decac5e6a59c130cc0"} Jul 17 13:48:05.673032 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:48:05.672946 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:49:10.976071 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:10.976037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:25.519677 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:25.519634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:49:25.520144 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:25.519808 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 17 13:49:25.520144 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:25.519873 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs podName:33042295-c3b5-46b6-bdb9-7d6c9095f83f nodeName:}" failed. No retries permitted until 2026-07-17 13:51:27.51985631 +0000 UTC m=+374.280721300 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs") pod "network-metrics-daemon-v2nz4" (UID: "33042295-c3b5-46b6-bdb9-7d6c9095f83f") : secret "metrics-daemon-secret" not found Jul 17 13:49:37.206287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:37.206211 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:37.307179 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:37.307149 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:38.958489 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:38.958460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:39.051328 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:39.051292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:41.383364 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.383330 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xk5wr"] Jul 17 13:49:41.386519 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.386499 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.393366 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.393346 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 13:49:41.393513 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.393498 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 13:49:41.394001 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.393983 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kldnd\"" Jul 17 13:49:41.394294 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.394278 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 13:49:41.394372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.394302 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 13:49:41.429114 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.429084 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xk5wr"] Jul 17 13:49:41.432301 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.432278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-crio-socket\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.432392 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.432328 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.432446 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.432406 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49jw\" (UniqueName: \"kubernetes.io/projected/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-api-access-f49jw\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.432481 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.432463 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-data-volume\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.432520 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.432502 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.493098 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.493059 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:41.533074 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-crio-socket\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533098 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533138 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49jw\" (UniqueName: \"kubernetes.io/projected/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-api-access-f49jw\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-data-volume\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533175 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-crio-socket\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.533258 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:41.533252 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 13:49:41.533520 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:41.533324 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 13:49:41.533520 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:41.533329 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:42.033307122 +0000 UTC m=+268.794172118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : configmap "kube-rbac-proxy" not found Jul 17 13:49:41.533520 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:41.533376 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:42.033359283 +0000 UTC m=+268.794224277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : secret "insights-runtime-extractor-tls" not found Jul 17 13:49:41.533624 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.533521 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-data-volume\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.551823 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.551790 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49jw\" (UniqueName: \"kubernetes.io/projected/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-api-access-f49jw\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:41.671807 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.671689 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:41.829134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.829102 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:41.911585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:41.911538 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:42.037268 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.037214 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:42.037476 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.037336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:42.037476 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:42.037346 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 13:49:42.037476 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:42.037415 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:43.037392938 +0000 UTC m=+269.798257947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : configmap "kube-rbac-proxy" not found Jul 17 13:49:42.037640 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:42.037507 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 13:49:42.037640 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:42.037579 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:43.037561237 +0000 UTC m=+269.798426230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : secret "insights-runtime-extractor-tls" not found Jul 17 13:49:42.534950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.534920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:42.667328 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.667299 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:42.823481 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.823376 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:42.924555 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:42.924509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:43.044196 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:43.044156 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:43.044379 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:43.044254 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:43.044379 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:43.044280 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 13:49:43.044379 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:43.044357 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:45.044342073 +0000 UTC m=+271.805207065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : configmap "kube-rbac-proxy" not found Jul 17 13:49:43.044526 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:43.044388 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 13:49:43.044526 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:43.044454 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:45.044440753 +0000 UTC m=+271.805305745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : secret "insights-runtime-extractor-tls" not found Jul 17 13:49:43.350370 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:43.350348 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rtgps_181882a0-faee-4bd6-bce1-e4baf72269f8/dns-node-resolver/0.log" Jul 17 13:49:44.121878 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:44.121843 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:44.149181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:44.149157 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bfpp_c56b3af1-0414-43b6-a26f-1c8c7ab26463/node-ca/0.log" Jul 17 13:49:44.183184 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:44.183153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:44.984778 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:44.984730 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:45.061033 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:45.060994 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:45.061215 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:45.061107 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:45.061215 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:45.061153 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 13:49:45.061215 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:45.061197 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 13:49:45.061346 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:45.061223 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:49.061207344 +0000 UTC m=+275.822072338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : secret "insights-runtime-extractor-tls" not found Jul 17 13:49:45.061346 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:45.061246 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:49.061231492 +0000 UTC m=+275.822096499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : configmap "kube-rbac-proxy" not found Jul 17 13:49:45.349102 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:45.349081 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rq7pc_01d2f7b7-1f92-4d12-99f1-f1524f347729/serve-healthcheck-canary/0.log" Jul 17 13:49:46.372735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:46.372694 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:46.473217 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:46.473182 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:46.574027 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:46.573992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:46.677559 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:46.677472 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:46.777559 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:46.777532 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:47.478703 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:47.478666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:47.877082 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:47.877054 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:48.133830 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:48.133738 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:48.280635 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:48.280602 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:48.682908 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:48.682877 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:48.875976 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:48.875945 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:49.076513 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:49.076468 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:49.095428 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:49.095394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:49.095555 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:49.095498 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:49.095555 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:49.095517 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 13:49:49.095645 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:49.095573 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:57.095557818 +0000 UTC m=+283.856422810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : configmap "kube-rbac-proxy" not found Jul 17 13:49:49.095645 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:49.095639 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 13:49:49.095730 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:49.095699 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls podName:0c3c34ed-7e58-463b-a19f-6bbbec0a20f9 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:57.095687168 +0000 UTC m=+283.856552159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xk5wr" (UID: "0c3c34ed-7e58-463b-a19f-6bbbec0a20f9") : secret "insights-runtime-extractor-tls" not found Jul 17 13:49:50.116834 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.116798 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:50.281034 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.281002 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:50.347597 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.347566 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:50.471527 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.471447 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:50.756669 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.756640 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:50.879685 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:50.879653 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:51.076666 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:51.076591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:52.070324 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:52.070291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:52.270178 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:52.270151 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:52.674635 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:52.674586 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:52.873680 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:52.873650 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:52.963286 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:52.963199 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:53.476198 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:53.476168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:53.876296 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:53.876254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:54.237667 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:54.237584 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" podUID="ac6513eb-e134-4479-a0c5-2ec3378bc7b1" Jul 17 13:49:54.237667 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:54.237616 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vfrhc" podUID="7fa85d62-bc6d-437b-a286-ed8ad35ae662" Jul 17 13:49:54.237667 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:54.237577 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" Jul 17 13:49:54.271412 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.271379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:54.478694 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.478664 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:54.588219 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.588191 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:49:54.588219 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.588210 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:49:54.588462 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.588389 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:49:54.606224 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:54.606198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:55.069030 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:55.068998 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:55.682107 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:55.682073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:55.874603 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:55.874571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:56.274635 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:56.274608 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:56.485397 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:56.485367 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:56.887425 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:56.887397 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:57.004267 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.004238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:57.072508 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.072484 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:57.160866 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.160774 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:57.161008 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.160910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:57.161392 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.161370 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:57.162987 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.162962 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c3c34ed-7e58-463b-a19f-6bbbec0a20f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xk5wr\" (UID: \"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9\") " pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:57.295226 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.295193 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xk5wr" Jul 17 13:49:57.423789 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.423690 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xk5wr"] Jul 17 13:49:57.426173 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:49:57.426150 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c3c34ed_7e58_463b_a19f_6bbbec0a20f9.slice/crio-92f6a19c4592ddeead5ea86e084f938cc510879d4958ea5377cef38aa80908f4 WatchSource:0}: Error finding container 92f6a19c4592ddeead5ea86e084f938cc510879d4958ea5377cef38aa80908f4: Status 404 returned error can't find the container with id 92f6a19c4592ddeead5ea86e084f938cc510879d4958ea5377cef38aa80908f4 Jul 17 13:49:57.598154 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:57.598115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xk5wr" event={"ID":"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9","Type":"ContainerStarted","Data":"92f6a19c4592ddeead5ea86e084f938cc510879d4958ea5377cef38aa80908f4"} Jul 17 13:49:58.169437 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.169390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:49:58.169944 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.169447 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:49:58.169944 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.169532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:49:58.172568 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.172522 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fa85d62-bc6d-437b-a286-ed8ad35ae662-metrics-tls\") pod \"dns-default-vfrhc\" (UID: \"7fa85d62-bc6d-437b-a286-ed8ad35ae662\") " pod="openshift-dns/dns-default-vfrhc" Jul 17 13:49:58.172568 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.172524 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac6513eb-e134-4479-a0c5-2ec3378bc7b1-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-j2q6g\" (UID: \"ac6513eb-e134-4479-a0c5-2ec3378bc7b1\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:49:58.172801 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.172536 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"image-registry-fdc484b8d-h4ght\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:49:58.192915 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.192872 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bwc6l\"" Jul 17 13:49:58.192915 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.192906 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cnxqs\"" Jul 17 13:49:58.193140 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.192881 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gl8vw\"" Jul 17 13:49:58.200116 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.200087 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" Jul 17 13:49:58.200287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.200261 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:49:58.200359 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.200316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:49:58.382675 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.382628 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vfrhc"] Jul 17 13:49:58.387219 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:49:58.387180 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa85d62_bc6d_437b_a286_ed8ad35ae662.slice/crio-58344e132941ce78385248e05af2199fb21c30d05da75259fe54256f2f7d798d WatchSource:0}: Error finding container 58344e132941ce78385248e05af2199fb21c30d05da75259fe54256f2f7d798d: Status 404 returned error can't find the container with id 58344e132941ce78385248e05af2199fb21c30d05da75259fe54256f2f7d798d Jul 17 13:49:58.398237 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.398205 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g"] Jul 17 13:49:58.401292 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:49:58.401262 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6513eb_e134_4479_a0c5_2ec3378bc7b1.slice/crio-5f5d60e6a52726570877113bde299cd939f52099f22923e1d91e2f0452d1bd39 WatchSource:0}: Error finding container 5f5d60e6a52726570877113bde299cd939f52099f22923e1d91e2f0452d1bd39: Status 404 returned error can't find the container with id 5f5d60e6a52726570877113bde299cd939f52099f22923e1d91e2f0452d1bd39 Jul 17 13:49:58.422234 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.422170 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:49:58.426227 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:49:58.426191 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6037aea9_7003_4e2d_a869_496dc44409a8.slice/crio-a24624565cc15185c32b481fbbe0aec75cdad91ef7d354eac80a859dff967010 WatchSource:0}: Error finding container a24624565cc15185c32b481fbbe0aec75cdad91ef7d354eac80a859dff967010: Status 404 returned error can't find the container with id a24624565cc15185c32b481fbbe0aec75cdad91ef7d354eac80a859dff967010 Jul 17 13:49:58.603717 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.603678 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" event={"ID":"6037aea9-7003-4e2d-a869-496dc44409a8","Type":"ContainerStarted","Data":"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa"} Jul 17 13:49:58.603909 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.603770 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" event={"ID":"6037aea9-7003-4e2d-a869-496dc44409a8","Type":"ContainerStarted","Data":"a24624565cc15185c32b481fbbe0aec75cdad91ef7d354eac80a859dff967010"} Jul 17 13:49:58.603909 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.603802 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:49:58.605296 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.605257 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfrhc" event={"ID":"7fa85d62-bc6d-437b-a286-ed8ad35ae662","Type":"ContainerStarted","Data":"58344e132941ce78385248e05af2199fb21c30d05da75259fe54256f2f7d798d"} Jul 17 13:49:58.606547 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.606528 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" event={"ID":"ac6513eb-e134-4479-a0c5-2ec3378bc7b1","Type":"ContainerStarted","Data":"5f5d60e6a52726570877113bde299cd939f52099f22923e1d91e2f0452d1bd39"} Jul 17 13:49:58.627931 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:58.627876 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" podStartSLOduration=284.627854001 podStartE2EDuration="4m44.627854001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:58.626548342 +0000 UTC m=+285.387413356" watchObservedRunningTime="2026-07-17 13:49:58.627854001 +0000 UTC m=+285.388719020" Jul 17 13:49:59.430173 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.430119 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" podUID="aeb65c5d-3792-4cd3-ae34-9403c7c2786d" containerName="addon-agent" probeResult="failure" output="Get \"http://10.132.0.6:8000/healthz\": dial tcp 10.132.0.6:8000: connect: connection refused" Jul 17 13:49:59.465225 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.464951 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" podUID="6f45038c-8853-4be2-8d2e-368eddeb996d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/healthz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jul 17 13:49:59.617186 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.617104 2555 generic.go:356] "Generic (PLEG): container finished" podID="6f45038c-8853-4be2-8d2e-368eddeb996d" containerID="41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61" exitCode=1 Jul 17 13:49:59.617907 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.617651 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerDied","Data":"41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61"} Jul 17 13:49:59.617907 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.617783 2555 scope.go:117] "RemoveContainer" containerID="77f558c8d55445443fb9081e050ca8fbe91b1966c2f7264aa7317c6e18031c3f" Jul 17 13:49:59.618135 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:49:59.618108 2555 scope.go:117] "RemoveContainer" containerID="41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61" Jul 17 13:49:59.618378 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:49:59.618306 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-86f86f6987-ln57n_open-cluster-management-agent-addon(6f45038c-8853-4be2-8d2e-368eddeb996d)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" podUID="6f45038c-8853-4be2-8d2e-368eddeb996d" Jul 17 13:50:00.265717 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:00.265683 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:50:00.622671 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:00.622596 2555 generic.go:356] "Generic (PLEG): container finished" podID="aeb65c5d-3792-4cd3-ae34-9403c7c2786d" containerID="1296b5c748ff653d99ef26d43a2116f445b8983626a937decac5e6a59c130cc0" exitCode=255 Jul 17 13:50:00.623127 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:00.622672 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerDied","Data":"1296b5c748ff653d99ef26d43a2116f445b8983626a937decac5e6a59c130cc0"} Jul 17 13:50:00.623127 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:00.623050 2555 scope.go:117] "RemoveContainer" containerID="41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61" Jul 17 13:50:00.623127 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:00.623122 2555 scope.go:117] "RemoveContainer" containerID="1296b5c748ff653d99ef26d43a2116f445b8983626a937decac5e6a59c130cc0" Jul 17 13:50:00.623293 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:50:00.623230 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-86f86f6987-ln57n_open-cluster-management-agent-addon(6f45038c-8853-4be2-8d2e-368eddeb996d)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" podUID="6f45038c-8853-4be2-8d2e-368eddeb996d" Jul 17 13:50:00.623293 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:50:00.623267 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-67666f755b-6ssst_open-cluster-management-agent-addon(aeb65c5d-3792-4cd3-ae34-9403c7c2786d)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" podUID="aeb65c5d-3792-4cd3-ae34-9403c7c2786d" Jul 17 13:50:02.318415 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:02.318381 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:02.575874 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:02.575855 2555 scope.go:117] "RemoveContainer" containerID="3448b1dd95bc3bce5a3d27c1754a9f2600c3209af38acce9bd24a0036ea879d0" Jul 17 13:50:03.323160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.323125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:03.640849 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.640737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xk5wr" event={"ID":"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9","Type":"ContainerStarted","Data":"456af18b3feac4f4bce96908f8235ac54d201d1d373c945c8495ad5d507d29e9"} Jul 17 13:50:03.642416 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.642387 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" event={"ID":"ac6513eb-e134-4479-a0c5-2ec3378bc7b1","Type":"ContainerStarted","Data":"9eacb69a1eb6bdcc8f775242b47ee11fcb9e53b31cba82141c16caf0a38928ce"} Jul 17 13:50:03.644417 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.644384 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfrhc" event={"ID":"7fa85d62-bc6d-437b-a286-ed8ad35ae662","Type":"ContainerStarted","Data":"f7ee5b681e0650e6195f9b9e921d79026c0e32fcc704855a755a58ca9bdb2668"} Jul 17 13:50:03.644562 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.644438 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfrhc" event={"ID":"7fa85d62-bc6d-437b-a286-ed8ad35ae662","Type":"ContainerStarted","Data":"55c7d5101e520472a3762b49cd1adcd6fd2129ffd71bd1037989ce96d040f22d"} Jul 17 13:50:03.644635 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.644559 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:50:03.660692 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.660632 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-j2q6g" podStartSLOduration=262.111279325 podStartE2EDuration="4m26.66061044s" totalImagesPullingTime="4.549331115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:03.659765162 +0000 UTC m=+290.420630175" watchObservedRunningTime="2026-07-17 13:50:03.66061044 +0000 UTC m=+290.421475454" Jul 17 13:50:03.678578 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:03.678526 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vfrhc" podStartSLOduration=251.114057311 podStartE2EDuration="4m15.678505085s" totalImagesPullingTime="4.564447774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:03.677645409 +0000 UTC m=+290.438510415" watchObservedRunningTime="2026-07-17 13:50:03.678505085 +0000 UTC m=+290.439370099" Jul 17 13:50:06.656217 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:06.656134 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xk5wr" event={"ID":"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9","Type":"ContainerStarted","Data":"0bdcc5f874847d4f58818d3a5bfb262868a5d0c412539911af6447a37e47ddc2"} Jul 17 13:50:07.088339 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:07.088305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:07.183099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:07.183067 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:07.282504 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:07.282473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:08.673433 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:50:08.673391 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-v2nz4" podUID="33042295-c3b5-46b6-bdb9-7d6c9095f83f" Jul 17 13:50:09.429850 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:09.429815 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" Jul 17 13:50:09.430214 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:09.430197 2555 scope.go:117] "RemoveContainer" containerID="1296b5c748ff653d99ef26d43a2116f445b8983626a937decac5e6a59c130cc0" Jul 17 13:50:09.464975 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:09.464941 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:50:09.465336 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:09.465322 2555 scope.go:117] "RemoveContainer" containerID="41ea7b7858e6cb60250fe6bc7e39f3c23dd85d37415e6a1300370beff7e89b61" Jul 17 13:50:10.674703 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.674669 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" event={"ID":"6f45038c-8853-4be2-8d2e-368eddeb996d","Type":"ContainerStarted","Data":"1f5d8ad3b5ad51f404377a221129a84f14123f8469b0d3bac01b8edf1e9fd5ad"} Jul 17 13:50:10.675126 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.674976 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:50:10.676554 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.676521 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67666f755b-6ssst" event={"ID":"aeb65c5d-3792-4cd3-ae34-9403c7c2786d","Type":"ContainerStarted","Data":"07eacbe5c592f821a91941622a5f83401b897706964461f1a03f40ab590a1e99"} Jul 17 13:50:10.676737 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.676722 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86f86f6987-ln57n" Jul 17 13:50:10.678442 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.678422 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xk5wr" event={"ID":"0c3c34ed-7e58-463b-a19f-6bbbec0a20f9","Type":"ContainerStarted","Data":"f057c2d3ea8b481354167376c6056955f4fdd8c5bacfbd1f9fbb2b7eeedb5ac2"} Jul 17 13:50:10.729313 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:10.729257 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xk5wr" podStartSLOduration=17.556257888 podStartE2EDuration="29.72923797s" totalImagesPullingTime="12.172980082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:49:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:10.728260839 +0000 UTC m=+297.489125852" watchObservedRunningTime="2026-07-17 13:50:10.72923797 +0000 UTC m=+297.490102987" Jul 17 13:50:11.581549 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:11.581517 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:11.679825 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:11.679788 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:11.758403 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:11.758372 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:12.078383 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:12.078344 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:13.586021 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:13.585992 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 13:50:13.649122 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:13.649092 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vfrhc" Jul 17 13:50:14.779181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:14.779145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:14.854255 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:14.854224 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:14.969420 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:14.969390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:16.609865 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:16.609830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:17.816347 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:17.816306 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:17.887054 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:17.887027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:18.204781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:18.204685 2555 patch_prober.go:28] interesting pod/image-registry-fdc484b8d-h4ght container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 17 13:50:18.204781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:18.204739 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 13:50:18.909734 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:18.909705 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:19.272961 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:19.272928 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:19.445095 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:19.445058 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" podUID="d2ce42e4-32eb-41ad-af5f-a309b312e472" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 17 13:50:19.622655 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:19.622582 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:50:19.806582 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:19.806548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:19.875888 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:19.875808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:20.672941 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:20.672910 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:50:22.586503 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.586466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:22.653698 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.653661 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f7wkf"] Jul 17 13:50:22.678294 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.678266 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:22.755926 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.755891 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:22.857251 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.857178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:22.958687 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:22.958656 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.000820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.000791 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.005028 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.005010 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 13:50:23.005420 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.005401 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 13:50:23.005943 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.005926 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 13:50:23.006211 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.006190 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bbfks\"" Jul 17 13:50:23.006357 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.006342 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 13:50:23.006430 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.006372 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 13:50:23.006487 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.006426 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 13:50:23.057590 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057590 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s72b4\" (UniqueName: \"kubernetes.io/projected/a69770f0-9243-45db-bbbe-76567977e50c-kube-api-access-s72b4\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057621 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057648 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-tls\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057674 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-root\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-metrics-client-ca\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057745 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-textfile\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.057826 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057792 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-sys\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.058064 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.057866 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-wtmp\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.058169 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.058151 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.158657 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158576 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-wtmp\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.158657 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158637 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.158892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s72b4\" (UniqueName: \"kubernetes.io/projected/a69770f0-9243-45db-bbbe-76567977e50c-kube-api-access-s72b4\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.158892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.158892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158781 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-wtmp\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.158892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158854 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-tls\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-root\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-metrics-client-ca\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158954 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-textfile\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158981 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-sys\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.158992 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-root\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.159058 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69770f0-9243-45db-bbbe-76567977e50c-sys\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159489 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.159464 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-accelerators-collector-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159575 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.159547 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69770f0-9243-45db-bbbe-76567977e50c-metrics-client-ca\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.159672 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.159643 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-textfile\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.160679 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.160658 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.161235 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.161209 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.161415 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.161394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69770f0-9243-45db-bbbe-76567977e50c-node-exporter-tls\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.167235 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.167215 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s72b4\" (UniqueName: \"kubernetes.io/projected/a69770f0-9243-45db-bbbe-76567977e50c-kube-api-access-s72b4\") pod \"node-exporter-f7wkf\" (UID: \"a69770f0-9243-45db-bbbe-76567977e50c\") " pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.258559 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.258525 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.310175 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.310138 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f7wkf" Jul 17 13:50:23.357017 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:50:23.356991 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69770f0_9243_45db_bbbe_76567977e50c.slice/crio-3ca4800260cf78d75954659641273eb4ab89f13f25ca181726d660dd1ab4a443 WatchSource:0}: Error finding container 3ca4800260cf78d75954659641273eb4ab89f13f25ca181726d660dd1ab4a443: Status 404 returned error can't find the container with id 3ca4800260cf78d75954659641273eb4ab89f13f25ca181726d660dd1ab4a443 Jul 17 13:50:23.358548 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.358532 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:50:23.594401 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.594370 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.674902 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.674875 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:23.717685 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:23.717659 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7wkf" event={"ID":"a69770f0-9243-45db-bbbe-76567977e50c","Type":"ContainerStarted","Data":"3ca4800260cf78d75954659641273eb4ab89f13f25ca181726d660dd1ab4a443"} Jul 17 13:50:24.141214 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:24.141130 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:24.239852 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:24.239799 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:24.340208 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:24.340177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:24.440846 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:24.440734 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:25.015179 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.015155 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:25.630626 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.630600 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:25.725027 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.724993 2555 generic.go:356] "Generic (PLEG): container finished" podID="a69770f0-9243-45db-bbbe-76567977e50c" containerID="9b44991c3fa13f99cebfa69479f48558aaa4e484e97983e09919538b0e01412b" exitCode=0 Jul 17 13:50:25.725206 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.725037 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7wkf" event={"ID":"a69770f0-9243-45db-bbbe-76567977e50c","Type":"ContainerDied","Data":"9b44991c3fa13f99cebfa69479f48558aaa4e484e97983e09919538b0e01412b"} Jul 17 13:50:25.727673 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.727651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:25.972580 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:25.972493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:26.084425 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.084394 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:26.730098 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.730065 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7wkf" event={"ID":"a69770f0-9243-45db-bbbe-76567977e50c","Type":"ContainerStarted","Data":"e470506afd9e1ad2cfd61f258ea826db7c54560774d7e099392dd102aaddb704"} Jul 17 13:50:26.730260 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.730115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f7wkf" event={"ID":"a69770f0-9243-45db-bbbe-76567977e50c","Type":"ContainerStarted","Data":"17725ef2891fd980359c973a69b80a00915ff20f15bd7bc94a2abdf8fcaae832"} Jul 17 13:50:26.750904 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.750859 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f7wkf" podStartSLOduration=3.001723375 podStartE2EDuration="4.75084323s" totalImagesPullingTime="1.6580467s" totalInitContainerRuntime="91.073155ms" isStatefulPod=true podCreationTimestamp="2026-07-17 13:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:26.749873092 +0000 UTC m=+313.510738105" watchObservedRunningTime="2026-07-17 13:50:26.75084323 +0000 UTC m=+313.511708242" Jul 17 13:50:26.898927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.898894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:26.977148 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:26.977112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:27.398087 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:27.398058 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:28.269141 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:28.269088 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:28.902111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:28.902078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:29.444724 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:29.444690 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" podUID="d2ce42e4-32eb-41ad-af5f-a309b312e472" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 17 13:50:29.713629 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:29.713549 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:50:29.812168 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:29.812135 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:29.882646 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:29.882617 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:29.984940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:29.984863 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:31.790843 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:31.790812 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:31.890642 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:31.890607 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:32.319623 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:32.319591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:32.420469 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:32.420432 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:36.886160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:36.886117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:36.974400 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:36.974369 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:37.042773 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:37.042722 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:37.173451 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:37.173349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:37.547520 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:37.547473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:37.652410 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:37.652380 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:37.962925 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:37.962835 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.022919 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.022887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.177429 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.177399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.279565 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.279535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.568020 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.567925 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.770098 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.770022 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:38.871946 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:38.871831 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:39.444385 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.444344 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" podUID="d2ce42e4-32eb-41ad-af5f-a309b312e472" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 17 13:50:39.444771 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.444429 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" Jul 17 13:50:39.444930 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.444900 2555 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4c4357d933fc0467eb1cb38b890fe70556cb4abdeda99aaba84d42bb416b7275"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 17 13:50:39.444968 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.444951 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" podUID="d2ce42e4-32eb-41ad-af5f-a309b312e472" containerName="service-proxy" containerID="cri-o://4c4357d933fc0467eb1cb38b890fe70556cb4abdeda99aaba84d42bb416b7275" gracePeriod=30 Jul 17 13:50:39.773029 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.772997 2555 generic.go:356] "Generic (PLEG): container finished" podID="d2ce42e4-32eb-41ad-af5f-a309b312e472" containerID="4c4357d933fc0467eb1cb38b890fe70556cb4abdeda99aaba84d42bb416b7275" exitCode=2 Jul 17 13:50:39.773209 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.773067 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerDied","Data":"4c4357d933fc0467eb1cb38b890fe70556cb4abdeda99aaba84d42bb416b7275"} Jul 17 13:50:39.773209 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.773109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-859c669d89-cf887" event={"ID":"d2ce42e4-32eb-41ad-af5f-a309b312e472","Type":"ContainerStarted","Data":"0a70e04e8622cffe0d5a9010503e4e3d1ba8d1a600393736a374413715a02bdc"} Jul 17 13:50:39.778759 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:39.778717 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:47.321741 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:47.321714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:47.384409 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:47.384373 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:50.996978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:50.996934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:51.063188 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:51.063153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:54.185300 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:54.185267 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:54.239629 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:54.239594 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:54.736431 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:54.736398 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" containerName="registry" containerID="cri-o://8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa" gracePeriod=30 Jul 17 13:50:54.985849 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:54.985827 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:50:55.095380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095325 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095388 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct7dw\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095421 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095447 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095484 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095506 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095522 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.095632 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.095550 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates\") pod \"6037aea9-7003-4e2d-a869-496dc44409a8\" (UID: \"6037aea9-7003-4e2d-a869-496dc44409a8\") " Jul 17 13:50:55.096134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.096067 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:50:55.096134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.096086 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:50:55.098129 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.098097 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:50:55.098247 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.098184 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw" (OuterVolumeSpecName: "kube-api-access-ct7dw") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "kube-api-access-ct7dw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:50:55.098247 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.098192 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:50:55.098368 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.098289 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:50:55.098368 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.098343 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:50:55.104079 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.104052 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6037aea9-7003-4e2d-a869-496dc44409a8" (UID: "6037aea9-7003-4e2d-a869-496dc44409a8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:50:55.196735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196695 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct7dw\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-kube-api-access-ct7dw\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.196735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196727 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6037aea9-7003-4e2d-a869-496dc44409a8-ca-trust-extracted\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.196735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196736 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-image-registry-private-configuration\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.197194 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196780 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-bound-sa-token\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.197194 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196790 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-trusted-ca\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.197194 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196798 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6037aea9-7003-4e2d-a869-496dc44409a8-installation-pull-secrets\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.197194 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196806 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6037aea9-7003-4e2d-a869-496dc44409a8-registry-certificates\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.197194 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.196814 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6037aea9-7003-4e2d-a869-496dc44409a8-registry-tls\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:50:55.826271 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.826237 2555 generic.go:356] "Generic (PLEG): container finished" podID="6037aea9-7003-4e2d-a869-496dc44409a8" containerID="8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa" exitCode=0 Jul 17 13:50:55.826460 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.826286 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" Jul 17 13:50:55.826460 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.826289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" event={"ID":"6037aea9-7003-4e2d-a869-496dc44409a8","Type":"ContainerDied","Data":"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa"} Jul 17 13:50:55.826460 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.826412 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdc484b8d-h4ght" event={"ID":"6037aea9-7003-4e2d-a869-496dc44409a8","Type":"ContainerDied","Data":"a24624565cc15185c32b481fbbe0aec75cdad91ef7d354eac80a859dff967010"} Jul 17 13:50:55.826460 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.826437 2555 scope.go:117] "RemoveContainer" containerID="8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa" Jul 17 13:50:55.837546 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.837524 2555 scope.go:117] "RemoveContainer" containerID="8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa" Jul 17 13:50:55.837921 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:50:55.837899 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa\": container with ID starting with 8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa not found: ID does not exist" containerID="8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa" Jul 17 13:50:55.837965 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.837931 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa"} err="failed to get container status \"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa\": rpc error: code = NotFound desc = could not find container \"8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa\": container with ID starting with 8170a191074f9a155d3df16e78b52626d9dcfc19fe82ca067c73026f9228c6aa not found: ID does not exist" Jul 17 13:50:55.842878 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.842855 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:50:55.846625 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.846606 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-fdc484b8d-h4ght"] Jul 17 13:50:55.975814 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:55.975721 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:56.035014 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:56.034973 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:57.677429 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:57.677395 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" path="/var/lib/kubelet/pods/6037aea9-7003-4e2d-a869-496dc44409a8/volumes" Jul 17 13:50:58.982014 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:58.981979 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:59.083209 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:50:59.083178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:02.329795 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:02.329763 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:02.430945 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:02.430920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:02.584634 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:02.584555 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:02.646796 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:02.646767 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:03.189798 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:03.189735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:03.249597 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:03.249571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:27.530363 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.530324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:51:27.532640 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.532617 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33042295-c3b5-46b6-bdb9-7d6c9095f83f-metrics-certs\") pod \"network-metrics-daemon-v2nz4\" (UID: \"33042295-c3b5-46b6-bdb9-7d6c9095f83f\") " pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:51:27.576138 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.576105 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnsb\"" Jul 17 13:51:27.584874 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.584853 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v2nz4" Jul 17 13:51:27.714253 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.714224 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v2nz4"] Jul 17 13:51:27.717579 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:51:27.717553 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33042295_c3b5_46b6_bdb9_7d6c9095f83f.slice/crio-e97158bb4a22388a2d2e2f72c9660c39026f2eb7459793c42b20919d4e2d1139 WatchSource:0}: Error finding container e97158bb4a22388a2d2e2f72c9660c39026f2eb7459793c42b20919d4e2d1139: Status 404 returned error can't find the container with id e97158bb4a22388a2d2e2f72c9660c39026f2eb7459793c42b20919d4e2d1139 Jul 17 13:51:27.935560 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:27.935464 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2nz4" event={"ID":"33042295-c3b5-46b6-bdb9-7d6c9095f83f","Type":"ContainerStarted","Data":"e97158bb4a22388a2d2e2f72c9660c39026f2eb7459793c42b20919d4e2d1139"} Jul 17 13:51:28.941037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:28.940950 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2nz4" event={"ID":"33042295-c3b5-46b6-bdb9-7d6c9095f83f","Type":"ContainerStarted","Data":"02f6443f106ae426f873dd88339cfa022d932634032dde05985ff9a8c2582f1a"} Jul 17 13:51:28.941037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:28.941001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v2nz4" event={"ID":"33042295-c3b5-46b6-bdb9-7d6c9095f83f","Type":"ContainerStarted","Data":"7d4996c15c65c7341d6d0a893616ad750c5ec7d8db4a45937ca07996b59476fa"} Jul 17 13:51:28.956555 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:28.956503 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v2nz4" podStartSLOduration=375.051229525 podStartE2EDuration="6m15.95648557s" totalImagesPullingTime="905.256045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:51:28.955929067 +0000 UTC m=+375.716794081" watchObservedRunningTime="2026-07-17 13:51:28.95648557 +0000 UTC m=+375.717350585" Jul 17 13:51:29.058061 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:29.058012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:29.127523 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:29.127490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:29.196642 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:29.196547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:31.172501 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:31.172459 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:31.239451 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:31.239420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:31.308782 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:31.308722 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:37.426270 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:37.426234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:37.481540 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:37.481512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:38.784432 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:38.784396 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:38.849196 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:38.849146 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:40.911139 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:40.911104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:40.972844 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:40.972814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:42.971296 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:42.971257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:50.814289 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:50.814238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:50.883043 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:50.883009 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:52.320323 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:52.320276 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:52.421848 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:52.421823 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:51:52.476951 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:51:52.476921 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:02.341810 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:02.341771 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:02.441761 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:02.441726 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:26.803032 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:26.802984 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:26.858505 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:26.858476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:40.223102 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:40.223017 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:40.322100 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:40.322064 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:52:41.057070 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.057033 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww"] Jul 17 13:52:41.057314 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.057302 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" containerName="registry" Jul 17 13:52:41.057356 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.057315 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" containerName="registry" Jul 17 13:52:41.057389 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.057357 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6037aea9-7003-4e2d-a869-496dc44409a8" containerName="registry" Jul 17 13:52:41.060201 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.060185 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.062693 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.062674 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 13:52:41.063761 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.063728 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 13:52:41.063892 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.063876 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m92l2\"" Jul 17 13:52:41.080135 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.080104 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww"] Jul 17 13:52:41.135953 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.135915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.135953 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.135957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqz4w\" (UniqueName: \"kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.136149 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.136008 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.237350 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.237313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.237781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.237365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.237781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.237392 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqz4w\" (UniqueName: \"kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.237781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.237709 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.237781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.237734 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.246940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.246907 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqz4w\" (UniqueName: \"kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.369671 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.369563 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:52:41.495963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:41.495932 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww"] Jul 17 13:52:41.499246 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:52:41.499219 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3899e40e_2a08_46d8_b0c7_eb03560d38ee.slice/crio-7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c WatchSource:0}: Error finding container 7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c: Status 404 returned error can't find the container with id 7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c Jul 17 13:52:42.179487 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:42.179441 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerStarted","Data":"7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c"} Jul 17 13:52:49.206816 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:49.206770 2555 generic.go:356] "Generic (PLEG): container finished" podID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerID="44aa436895c005145608228abd0afad47d95a1eaad1e3e4a35fe3b72ffa82832" exitCode=0 Jul 17 13:52:49.207202 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:49.206836 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerDied","Data":"44aa436895c005145608228abd0afad47d95a1eaad1e3e4a35fe3b72ffa82832"} Jul 17 13:52:52.221898 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:52.221861 2555 generic.go:356] "Generic (PLEG): container finished" podID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerID="e00befac3d61525af6ea828cb12a31386422786d8ff3bb33f8794793b598a099" exitCode=0 Jul 17 13:52:52.222313 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:52.221906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerDied","Data":"e00befac3d61525af6ea828cb12a31386422786d8ff3bb33f8794793b598a099"} Jul 17 13:52:59.249177 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:59.249139 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerStarted","Data":"2cb9effb50569a5ecb942aca1295a88614f15fbde36e60f64096189746057f7f"} Jul 17 13:52:59.269671 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:52:59.269607 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" podStartSLOduration=2.491004893 podStartE2EDuration="18.269588469s" totalImagesPullingTime="15.778583576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:52:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:52:59.269050937 +0000 UTC m=+466.029915950" watchObservedRunningTime="2026-07-17 13:52:59.269588469 +0000 UTC m=+466.030453511" Jul 17 13:53:00.254689 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:00.254657 2555 generic.go:356] "Generic (PLEG): container finished" podID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerID="2cb9effb50569a5ecb942aca1295a88614f15fbde36e60f64096189746057f7f" exitCode=0 Jul 17 13:53:00.254689 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:00.254694 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerDied","Data":"2cb9effb50569a5ecb942aca1295a88614f15fbde36e60f64096189746057f7f"} Jul 17 13:53:01.379608 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.379581 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:53:01.502921 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.502880 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util\") pod \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " Jul 17 13:53:01.503117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.502942 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqz4w\" (UniqueName: \"kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w\") pod \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " Jul 17 13:53:01.503117 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.502985 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle\") pod \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\" (UID: \"3899e40e-2a08-46d8-b0c7-eb03560d38ee\") " Jul 17 13:53:01.503642 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.503617 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle" (OuterVolumeSpecName: "bundle") pod "3899e40e-2a08-46d8-b0c7-eb03560d38ee" (UID: "3899e40e-2a08-46d8-b0c7-eb03560d38ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:01.505186 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.505162 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w" (OuterVolumeSpecName: "kube-api-access-fqz4w") pod "3899e40e-2a08-46d8-b0c7-eb03560d38ee" (UID: "3899e40e-2a08-46d8-b0c7-eb03560d38ee"). InnerVolumeSpecName "kube-api-access-fqz4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:53:01.507461 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.507434 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util" (OuterVolumeSpecName: "util") pod "3899e40e-2a08-46d8-b0c7-eb03560d38ee" (UID: "3899e40e-2a08-46d8-b0c7-eb03560d38ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:01.603602 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.603510 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:01.603602 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.603542 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqz4w\" (UniqueName: \"kubernetes.io/projected/3899e40e-2a08-46d8-b0c7-eb03560d38ee-kube-api-access-fqz4w\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:01.603602 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:01.603554 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3899e40e-2a08-46d8-b0c7-eb03560d38ee-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:02.263738 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:02.263710 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" Jul 17 13:53:02.263912 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:02.263710 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408wjww" event={"ID":"3899e40e-2a08-46d8-b0c7-eb03560d38ee","Type":"ContainerDied","Data":"7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c"} Jul 17 13:53:02.263912 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:02.263797 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7005cd2489b9cb81a22b5d62db25d83e604aca0c715356aa6041b7c4f5f4334c" Jul 17 13:53:07.221322 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:07.221286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:08.046485 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:08.046450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:09.639657 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639626 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v"] Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639867 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="pull" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639878 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="pull" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639886 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="util" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639891 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="util" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639899 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="extract" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639904 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="extract" Jul 17 13:53:09.640052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.639946 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3899e40e-2a08-46d8-b0c7-eb03560d38ee" containerName="extract" Jul 17 13:53:09.642735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.642719 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.646283 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.646262 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:53:09.646412 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.646267 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 17 13:53:09.646412 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.646382 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tv725\"" Jul 17 13:53:09.662189 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.662161 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v"] Jul 17 13:53:09.760498 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.760451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.760498 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.760496 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzgd\" (UniqueName: \"kubernetes.io/projected/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-kube-api-access-cqzgd\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.861051 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.861017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzgd\" (UniqueName: \"kubernetes.io/projected/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-kube-api-access-cqzgd\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.861229 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.861088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.861410 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.861394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.871963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.871935 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzgd\" (UniqueName: \"kubernetes.io/projected/0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1-kube-api-access-cqzgd\") pod \"cert-manager-operator-controller-manager-55c444bf86-khq4v\" (UID: \"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:09.951038 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:09.950955 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" Jul 17 13:53:10.082545 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:10.082512 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v"] Jul 17 13:53:10.088313 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:53:10.088282 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7d6cbd_b0df_4922_8a62_7e1ecb1234c1.slice/crio-fcf6e34f6886f16ef2777ab693536ed9e19a0ed74f495bbeeaebfccc6189502e WatchSource:0}: Error finding container fcf6e34f6886f16ef2777ab693536ed9e19a0ed74f495bbeeaebfccc6189502e: Status 404 returned error can't find the container with id fcf6e34f6886f16ef2777ab693536ed9e19a0ed74f495bbeeaebfccc6189502e Jul 17 13:53:10.291564 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:10.291473 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" event={"ID":"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1","Type":"ContainerStarted","Data":"fcf6e34f6886f16ef2777ab693536ed9e19a0ed74f495bbeeaebfccc6189502e"} Jul 17 13:53:11.260038 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:11.260010 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:11.358934 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:11.358893 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:12.302660 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:12.302588 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" event={"ID":"0b7d6cbd-b0df-4922-8a62-7e1ecb1234c1","Type":"ContainerStarted","Data":"8fa7ef8c71c1dff23cbea589b4aa0468bc3e61ae21d5581bafcf0cded46369de"} Jul 17 13:53:12.326401 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:12.323420 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-khq4v" podStartSLOduration=1.693211354 podStartE2EDuration="3.323397278s" totalImagesPullingTime="1.630185924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:53:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:53:12.323190177 +0000 UTC m=+479.084055194" watchObservedRunningTime="2026-07-17 13:53:12.323397278 +0000 UTC m=+479.084262289" Jul 17 13:53:12.729398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:12.729315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:12.887895 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:12.887860 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:13.050738 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.050703 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd"] Jul 17 13:53:13.054011 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.053992 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.057064 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.057044 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 13:53:13.057723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.057706 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m92l2\"" Jul 17 13:53:13.057807 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.057703 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 13:53:13.072277 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.072246 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd"] Jul 17 13:53:13.186977 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.186923 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sm9q\" (UniqueName: \"kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.187166 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.187042 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.187166 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.187088 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.287547 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.287507 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.287669 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.287553 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.287712 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.287676 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sm9q\" (UniqueName: \"kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.287947 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.287929 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.287993 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.287958 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.297950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.297923 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sm9q\" (UniqueName: \"kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.363319 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.363240 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:13.476132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.476098 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:13.500167 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.500141 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd"] Jul 17 13:53:13.502347 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:53:13.502318 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eca2a5f_8878_49a3_925b_4f5c9adc9e78.slice/crio-00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497 WatchSource:0}: Error finding container 00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497: Status 404 returned error can't find the container with id 00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497 Jul 17 13:53:13.584716 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:13.584684 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:14.311200 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:14.311167 2555 generic.go:356] "Generic (PLEG): container finished" podID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerID="a0dc37aa5e12d3e63513062a9b18ac68666839707aebd613668adf709f70033d" exitCode=0 Jul 17 13:53:14.311325 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:14.311229 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" event={"ID":"5eca2a5f-8878-49a3-925b-4f5c9adc9e78","Type":"ContainerDied","Data":"a0dc37aa5e12d3e63513062a9b18ac68666839707aebd613668adf709f70033d"} Jul 17 13:53:14.311325 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:14.311279 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" event={"ID":"5eca2a5f-8878-49a3-925b-4f5c9adc9e78","Type":"ContainerStarted","Data":"00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497"} Jul 17 13:53:15.685322 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:15.685290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:15.891920 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:15.891887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:17.324349 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:17.324312 2555 generic.go:356] "Generic (PLEG): container finished" podID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerID="c895c59fd68c55101dd2887f42a3da43a9cea8a26077d4257a4b0483a5d74add" exitCode=0 Jul 17 13:53:17.324705 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:17.324391 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" event={"ID":"5eca2a5f-8878-49a3-925b-4f5c9adc9e78","Type":"ContainerDied","Data":"c895c59fd68c55101dd2887f42a3da43a9cea8a26077d4257a4b0483a5d74add"} Jul 17 13:53:17.999103 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:17.999071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:18.082973 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:18.082944 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:18.329720 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:18.329637 2555 generic.go:356] "Generic (PLEG): container finished" podID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerID="5cf014178117626683ea0b3dbc53c18c4e2b9ad66018d5a6a162569879092c36" exitCode=0 Jul 17 13:53:18.330082 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:18.329717 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" event={"ID":"5eca2a5f-8878-49a3-925b-4f5c9adc9e78","Type":"ContainerDied","Data":"5cf014178117626683ea0b3dbc53c18c4e2b9ad66018d5a6a162569879092c36"} Jul 17 13:53:19.457641 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.457611 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:19.636144 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.636060 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sm9q\" (UniqueName: \"kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q\") pod \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " Jul 17 13:53:19.636144 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.636094 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle\") pod \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " Jul 17 13:53:19.636144 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.636121 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util\") pod \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\" (UID: \"5eca2a5f-8878-49a3-925b-4f5c9adc9e78\") " Jul 17 13:53:19.636551 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.636528 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle" (OuterVolumeSpecName: "bundle") pod "5eca2a5f-8878-49a3-925b-4f5c9adc9e78" (UID: "5eca2a5f-8878-49a3-925b-4f5c9adc9e78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:19.638257 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.638229 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q" (OuterVolumeSpecName: "kube-api-access-8sm9q") pod "5eca2a5f-8878-49a3-925b-4f5c9adc9e78" (UID: "5eca2a5f-8878-49a3-925b-4f5c9adc9e78"). InnerVolumeSpecName "kube-api-access-8sm9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:53:19.640914 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.640886 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util" (OuterVolumeSpecName: "util") pod "5eca2a5f-8878-49a3-925b-4f5c9adc9e78" (UID: "5eca2a5f-8878-49a3-925b-4f5c9adc9e78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:19.737132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.737102 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:19.737132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.737129 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sm9q\" (UniqueName: \"kubernetes.io/projected/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-kube-api-access-8sm9q\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:19.737335 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:19.737141 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5eca2a5f-8878-49a3-925b-4f5c9adc9e78-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:20.339080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:20.339042 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" event={"ID":"5eca2a5f-8878-49a3-925b-4f5c9adc9e78","Type":"ContainerDied","Data":"00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497"} Jul 17 13:53:20.339080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:20.339076 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhfgvd" Jul 17 13:53:20.339080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:20.339088 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00617c181a6a9fd8d2ecdba05ced04607fc01a9db300dcbf3986a5c21d457497" Jul 17 13:53:22.321945 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:22.321907 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:22.434985 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:22.434947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:23.152612 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:23.152575 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:23.893020 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:23.892988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:26.007763 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:26.007714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:26.167511 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:26.167480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:31.419093 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:31.419056 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:31.569490 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:31.569451 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:33.281910 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.281879 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk"] Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282098 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="pull" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282108 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="pull" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282122 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="util" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282128 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="util" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282135 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="extract" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282140 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="extract" Jul 17 13:53:33.282275 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.282185 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eca2a5f-8878-49a3-925b-4f5c9adc9e78" containerName="extract" Jul 17 13:53:33.287821 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.287802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.290415 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.290389 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 13:53:33.290529 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.290397 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m92l2\"" Jul 17 13:53:33.290529 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.290479 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 13:53:33.294101 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.294076 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk"] Jul 17 13:53:33.442071 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.442036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.442244 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.442094 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.442244 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.442149 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd6v6\" (UniqueName: \"kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.531314 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.531278 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:33.543533 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.543471 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.543533 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.543513 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd6v6\" (UniqueName: \"kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.543676 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.543552 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.543908 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.543878 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.543954 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.543935 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.552246 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.552216 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd6v6\" (UniqueName: \"kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.596951 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.596912 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:33.723285 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:33.723250 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk"] Jul 17 13:53:33.729909 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:53:33.729880 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb574b273_14fc_4b22_8e25_2dfdcc76f356.slice/crio-bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f WatchSource:0}: Error finding container bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f: Status 404 returned error can't find the container with id bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f Jul 17 13:53:34.394969 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:34.394935 2555 generic.go:356] "Generic (PLEG): container finished" podID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerID="7db3909d13b55cbc03b01a9b07383acab39f43d89299793fbb243cb944a7ddbd" exitCode=0 Jul 17 13:53:34.395369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:34.395003 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" event={"ID":"b574b273-14fc-4b22-8e25-2dfdcc76f356","Type":"ContainerDied","Data":"7db3909d13b55cbc03b01a9b07383acab39f43d89299793fbb243cb944a7ddbd"} Jul 17 13:53:34.395369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:34.395042 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" event={"ID":"b574b273-14fc-4b22-8e25-2dfdcc76f356","Type":"ContainerStarted","Data":"bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f"} Jul 17 13:53:35.400956 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:35.400867 2555 generic.go:356] "Generic (PLEG): container finished" podID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerID="f2db46352e5648d4ad3cafac26c0f92e083b159ad98dbe35fe2d2be3e0ed7791" exitCode=0 Jul 17 13:53:35.400956 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:35.400913 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" event={"ID":"b574b273-14fc-4b22-8e25-2dfdcc76f356","Type":"ContainerDied","Data":"f2db46352e5648d4ad3cafac26c0f92e083b159ad98dbe35fe2d2be3e0ed7791"} Jul 17 13:53:36.406903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:36.406868 2555 generic.go:356] "Generic (PLEG): container finished" podID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerID="9e58dfa06dc75a43761e852b79d5c8dd17dbe402fb25703c61e3a13774c4f228" exitCode=0 Jul 17 13:53:36.407278 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:36.406923 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" event={"ID":"b574b273-14fc-4b22-8e25-2dfdcc76f356","Type":"ContainerDied","Data":"9e58dfa06dc75a43761e852b79d5c8dd17dbe402fb25703c61e3a13774c4f228"} Jul 17 13:53:37.543875 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.543852 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:37.572049 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.572020 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle\") pod \"b574b273-14fc-4b22-8e25-2dfdcc76f356\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " Jul 17 13:53:37.572204 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.572064 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd6v6\" (UniqueName: \"kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6\") pod \"b574b273-14fc-4b22-8e25-2dfdcc76f356\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " Jul 17 13:53:37.572204 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.572113 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util\") pod \"b574b273-14fc-4b22-8e25-2dfdcc76f356\" (UID: \"b574b273-14fc-4b22-8e25-2dfdcc76f356\") " Jul 17 13:53:37.572829 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.572803 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle" (OuterVolumeSpecName: "bundle") pod "b574b273-14fc-4b22-8e25-2dfdcc76f356" (UID: "b574b273-14fc-4b22-8e25-2dfdcc76f356"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:37.574165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.574140 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6" (OuterVolumeSpecName: "kube-api-access-wd6v6") pod "b574b273-14fc-4b22-8e25-2dfdcc76f356" (UID: "b574b273-14fc-4b22-8e25-2dfdcc76f356"). InnerVolumeSpecName "kube-api-access-wd6v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:53:37.579268 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.579236 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util" (OuterVolumeSpecName: "util") pod "b574b273-14fc-4b22-8e25-2dfdcc76f356" (UID: "b574b273-14fc-4b22-8e25-2dfdcc76f356"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:53:37.672933 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.672837 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:37.672933 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.672871 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd6v6\" (UniqueName: \"kubernetes.io/projected/b574b273-14fc-4b22-8e25-2dfdcc76f356-kube-api-access-wd6v6\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:37.672933 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:37.672884 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b574b273-14fc-4b22-8e25-2dfdcc76f356-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:53:38.415966 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:38.415932 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" event={"ID":"b574b273-14fc-4b22-8e25-2dfdcc76f356","Type":"ContainerDied","Data":"bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f"} Jul 17 13:53:38.415966 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:38.415961 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bbhk" Jul 17 13:53:38.416182 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:38.415984 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bad0e06303a7521cbef1e13f6cb785b5533ff394db2d7443b7b517264626818f" Jul 17 13:53:47.832483 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:47.832435 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:47.985467 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:47.985436 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:48.482224 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.482195 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:48.821400 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.821366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:48.940940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.940911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:48.991480 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991443 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v"] Jul 17 13:53:48.991678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991667 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="util" Jul 17 13:53:48.991723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991680 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="util" Jul 17 13:53:48.991723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991698 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="extract" Jul 17 13:53:48.991723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991704 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="extract" Jul 17 13:53:48.991723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991712 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="pull" Jul 17 13:53:48.991723 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991717 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="pull" Jul 17 13:53:48.991886 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.991777 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b574b273-14fc-4b22-8e25-2dfdcc76f356" containerName="extract" Jul 17 13:53:48.994951 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.994933 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:48.998243 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.998218 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 13:53:48.998372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.998279 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pxqwq\"" Jul 17 13:53:48.998372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.998332 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 13:53:48.998489 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.998422 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 13:53:48.998631 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:48.998614 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 13:53:49.029981 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.029952 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v"] Jul 17 13:53:49.048631 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.048596 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.048631 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.048641 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.048854 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.048661 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.048854 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.048731 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q6wl\" (UniqueName: \"kubernetes.io/projected/37282188-cdde-4775-9f2e-c25c9b1a95e0-kube-api-access-8q6wl\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.048854 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.048775 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.149889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.149788 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q6wl\" (UniqueName: \"kubernetes.io/projected/37282188-cdde-4775-9f2e-c25c9b1a95e0-kube-api-access-8q6wl\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.149889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.149832 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.149889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.149859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.149889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.149894 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.150205 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.149916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.150331 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.150308 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.152378 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.152347 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37282188-cdde-4775-9f2e-c25c9b1a95e0-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.152493 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.152421 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.152493 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.152453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37282188-cdde-4775-9f2e-c25c9b1a95e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.159534 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.159506 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q6wl\" (UniqueName: \"kubernetes.io/projected/37282188-cdde-4775-9f2e-c25c9b1a95e0-kube-api-access-8q6wl\") pod \"opendatahub-operator-controller-manager-57f79678b4-f5d4v\" (UID: \"37282188-cdde-4775-9f2e-c25c9b1a95e0\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.276184 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.276152 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.304865 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.304828 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:49.375454 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.375425 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.441519 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.441436 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v"] Jul 17 13:53:49.445023 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:53:49.444999 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37282188_cdde_4775_9f2e_c25c9b1a95e0.slice/crio-6830fcbc572768d265302f5dfae3e30260b3492ac581aaadb21e296a7935f7dd WatchSource:0}: Error finding container 6830fcbc572768d265302f5dfae3e30260b3492ac581aaadb21e296a7935f7dd: Status 404 returned error can't find the container with id 6830fcbc572768d265302f5dfae3e30260b3492ac581aaadb21e296a7935f7dd Jul 17 13:53:49.452050 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.452016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.459116 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.459091 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" event={"ID":"37282188-cdde-4775-9f2e-c25c9b1a95e0","Type":"ContainerStarted","Data":"6830fcbc572768d265302f5dfae3e30260b3492ac581aaadb21e296a7935f7dd"} Jul 17 13:53:49.520037 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.519989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.620874 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.620842 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.728070 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.727969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:49.823176 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:49.823143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:53:52.474284 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:52.474256 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" event={"ID":"37282188-cdde-4775-9f2e-c25c9b1a95e0","Type":"ContainerStarted","Data":"4d15b8af1c822e13e22e6cf2eef1781ee346500261d92feba29cf8d7a05d7e70"} Jul 17 13:53:54.483228 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:54.483194 2555 generic.go:356] "Generic (PLEG): container finished" podID="37282188-cdde-4775-9f2e-c25c9b1a95e0" containerID="4d15b8af1c822e13e22e6cf2eef1781ee346500261d92feba29cf8d7a05d7e70" exitCode=0 Jul 17 13:53:54.483622 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:54.483266 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" event={"ID":"37282188-cdde-4775-9f2e-c25c9b1a95e0","Type":"ContainerDied","Data":"4d15b8af1c822e13e22e6cf2eef1781ee346500261d92feba29cf8d7a05d7e70"} Jul 17 13:53:55.489453 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:55.489418 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" event={"ID":"37282188-cdde-4775-9f2e-c25c9b1a95e0","Type":"ContainerStarted","Data":"e82ae05eda5655c0c84ff4188314ca240749c849887b2be714f90e70ade1fc25"} Jul 17 13:53:55.489822 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:55.489634 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:53:55.526221 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:53:55.526172 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" podStartSLOduration=3.133179275 podStartE2EDuration="7.526153723s" totalImagesPullingTime="3.297329686s" totalInitContainerRuntime="1.095644762s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:53:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:53:55.525831884 +0000 UTC m=+522.286696898" watchObservedRunningTime="2026-07-17 13:53:55.526153723 +0000 UTC m=+522.287018735" Jul 17 13:54:04.023158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:04.023124 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:04.125017 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:04.124979 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:06.020330 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.020252 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q"] Jul 17 13:54:06.033742 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.033717 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.036366 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.036337 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q"] Jul 17 13:54:06.036961 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.036939 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 17 13:54:06.037093 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.037074 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 17 13:54:06.037158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.036939 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jpqgw\"" Jul 17 13:54:06.037158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.036944 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 17 13:54:06.037256 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.036944 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 17 13:54:06.037256 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.037213 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:54:06.081160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.081122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.081351 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.081165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jmqm\" (UniqueName: \"kubernetes.io/projected/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-kube-api-access-6jmqm\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.081351 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.081214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-metrics-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.081351 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.081268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-manager-config\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.181675 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.181634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-metrics-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.181908 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.181688 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-manager-config\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.181908 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.181790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.181908 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.181809 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmqm\" (UniqueName: \"kubernetes.io/projected/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-kube-api-access-6jmqm\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.182507 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.182485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-manager-config\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.184170 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.184146 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.184271 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.184231 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-metrics-cert\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.189703 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.189682 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmqm\" (UniqueName: \"kubernetes.io/projected/65d7285f-6dec-477a-b2f4-bfc0989f2ea5-kube-api-access-6jmqm\") pod \"lws-controller-manager-77668bc899-kmt6q\" (UID: \"65d7285f-6dec-477a-b2f4-bfc0989f2ea5\") " pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.343428 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.343340 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:06.476624 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.476595 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q"] Jul 17 13:54:06.480119 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:54:06.480089 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d7285f_6dec_477a_b2f4_bfc0989f2ea5.slice/crio-cd872629656d23d70653fce009f691e25edfb483057a3b65736f24fc2c8b262e WatchSource:0}: Error finding container cd872629656d23d70653fce009f691e25edfb483057a3b65736f24fc2c8b262e: Status 404 returned error can't find the container with id cd872629656d23d70653fce009f691e25edfb483057a3b65736f24fc2c8b262e Jul 17 13:54:06.494938 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.494915 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-f5d4v" Jul 17 13:54:06.534062 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:06.534032 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" event={"ID":"65d7285f-6dec-477a-b2f4-bfc0989f2ea5","Type":"ContainerStarted","Data":"cd872629656d23d70653fce009f691e25edfb483057a3b65736f24fc2c8b262e"} Jul 17 13:54:08.151766 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.151727 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:08.302134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.302099 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:08.543847 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.543817 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" event={"ID":"65d7285f-6dec-477a-b2f4-bfc0989f2ea5","Type":"ContainerStarted","Data":"b8f7aa2dcdc13ef560102e72ed278c4e859796f0136bb9e77550f96d132c8d02"} Jul 17 13:54:08.544021 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.543926 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:08.579044 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.578946 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" podStartSLOduration=2.133742031 podStartE2EDuration="3.578928712s" totalImagesPullingTime="1.445186681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:08.578022883 +0000 UTC m=+535.338887910" watchObservedRunningTime="2026-07-17 13:54:08.578928712 +0000 UTC m=+535.339793726" Jul 17 13:54:08.971647 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:08.971565 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:09.083471 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:09.083437 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:09.268244 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:09.268210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:09.625855 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:09.625780 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:10.240259 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:10.240229 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:11.626258 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.626218 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:11.768104 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.768073 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t"] Jul 17 13:54:11.771327 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.771306 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.773661 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.773633 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 13:54:11.773775 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.773695 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-d5c64\"" Jul 17 13:54:11.774391 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.774372 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 13:54:11.774504 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.774411 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 13:54:11.774504 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.774419 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 13:54:11.784364 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.784342 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t"] Jul 17 13:54:11.824042 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.824010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tmp\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.824198 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.824058 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pln6x\" (UniqueName: \"kubernetes.io/projected/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-kube-api-access-pln6x\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.824198 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.824108 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.924820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.924727 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.924820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.924791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tmp\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.925013 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.924827 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pln6x\" (UniqueName: \"kubernetes.io/projected/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-kube-api-access-pln6x\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.925013 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:54:11.924879 2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 17 13:54:11.925013 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:54:11.924960 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs podName:ac4f2a5b-43bb-4e14-a62c-f8a19e93062e nodeName:}" failed. No retries permitted until 2026-07-17 13:54:12.4249392 +0000 UTC m=+539.185804201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs") pod "kube-auth-proxy-66c48b9c48-tp54t" (UID: "ac4f2a5b-43bb-4e14-a62c-f8a19e93062e") : secret "kube-auth-proxy-tls" not found Jul 17 13:54:11.927039 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.927015 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tmp\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.933022 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.933000 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pln6x\" (UniqueName: \"kubernetes.io/projected/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-kube-api-access-pln6x\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:11.998867 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:11.998833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:12.428573 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.428518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:12.430878 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.430859 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac4f2a5b-43bb-4e14-a62c-f8a19e93062e-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-tp54t\" (UID: \"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:12.681364 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.681258 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" Jul 17 13:54:12.815859 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.815736 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t"] Jul 17 13:54:12.818434 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:54:12.818407 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4f2a5b_43bb_4e14_a62c_f8a19e93062e.slice/crio-3a0db9a3e4cd377618d130b82a4ef709176bce744d4428b06e47dd70ee364da5 WatchSource:0}: Error finding container 3a0db9a3e4cd377618d130b82a4ef709176bce744d4428b06e47dd70ee364da5: Status 404 returned error can't find the container with id 3a0db9a3e4cd377618d130b82a4ef709176bce744d4428b06e47dd70ee364da5 Jul 17 13:54:12.895487 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.895452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:12.962212 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:12.962130 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:13.566819 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:13.566781 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" event={"ID":"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e","Type":"ContainerStarted","Data":"3a0db9a3e4cd377618d130b82a4ef709176bce744d4428b06e47dd70ee364da5"} Jul 17 13:54:13.997383 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:13.997349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:15.505385 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:15.505356 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 13:54:15.580176 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:15.580138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" event={"ID":"ac4f2a5b-43bb-4e14-a62c-f8a19e93062e","Type":"ContainerStarted","Data":"2f53c02df247846376d4ce1ad440815770b9fb2aaa5c90d8a2cd524793ee6471"} Jul 17 13:54:15.598833 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:15.598785 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-tp54t" podStartSLOduration=1.916326674 podStartE2EDuration="4.598768556s" totalImagesPullingTime="2.682441882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:15.596973623 +0000 UTC m=+542.357838637" watchObservedRunningTime="2026-07-17 13:54:15.598768556 +0000 UTC m=+542.359633569" Jul 17 13:54:17.715701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:17.715666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:17.822019 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:17.821988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:19.124585 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:19.124553 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:19.409002 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:19.408927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:19.550848 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:19.550818 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77668bc899-kmt6q" Jul 17 13:54:19.962994 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:19.962963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:21.312942 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:21.312914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:21.515340 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:21.515307 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:21.714058 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:21.713972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:21.923806 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:21.923746 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:22.759297 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:22.759270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:22.859313 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:22.859286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:22.991396 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:22.991363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:54:39.025995 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:54:39.025950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:00.055701 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:00.055664 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:09.941783 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:09.941731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:11.463369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:11.463334 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:25.687223 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.687184 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh"] Jul 17 13:55:25.690241 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.690223 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.692611 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.692588 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 13:55:25.692735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.692612 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gvwvs\"" Jul 17 13:55:25.692735 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.692614 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 13:55:25.698743 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.698719 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh"] Jul 17 13:55:25.770740 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.770699 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8hbk\" (UniqueName: \"kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.770943 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.770780 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.770943 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.770806 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.872067 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.872037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8hbk\" (UniqueName: \"kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.872244 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.872083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.872244 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.872205 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.872519 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.872497 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.872560 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.872508 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.882125 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.882099 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8hbk\" (UniqueName: \"kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:25.999593 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:25.999551 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:26.095464 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.095431 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7"] Jul 17 13:55:26.098882 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.098859 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.112927 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.112889 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7"] Jul 17 13:55:26.134326 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.134162 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh"] Jul 17 13:55:26.137099 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:55:26.137071 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d566442_5ddb_47a0_8405_dd2e3d00d3bb.slice/crio-b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2 WatchSource:0}: Error finding container b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2: Status 404 returned error can't find the container with id b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2 Jul 17 13:55:26.175180 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.175151 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.175288 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.175198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfvt\" (UniqueName: \"kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.175361 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.175312 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.275921 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.275837 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.275921 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.275880 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.275921 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.275917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfvt\" (UniqueName: \"kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.276270 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.276251 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.276309 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.276260 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.284298 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.284271 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfvt\" (UniqueName: \"kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.409660 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.409621 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:26.486551 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.486521 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8"] Jul 17 13:55:26.489996 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.489974 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.499305 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.499119 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8"] Jul 17 13:55:26.540852 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.540813 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7"] Jul 17 13:55:26.543297 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:55:26.543260 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc38f750_3f04_477b_8ff7_dc84a8c4fafa.slice/crio-b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90 WatchSource:0}: Error finding container b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90: Status 404 returned error can't find the container with id b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90 Jul 17 13:55:26.577972 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.577945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phtrp\" (UniqueName: \"kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.578106 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.577996 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.578106 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.578060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.679313 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.679273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phtrp\" (UniqueName: \"kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.679473 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.679334 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.679473 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.679367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.679726 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.679705 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.679785 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.679714 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.688571 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.688547 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phtrp\" (UniqueName: \"kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.801780 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.801680 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:26.875159 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.875128 2555 generic.go:356] "Generic (PLEG): container finished" podID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerID="def0020eb47494aaaad0ebfd1b5746afddabb48d99ff4c720369d2ed51db3d36" exitCode=0 Jul 17 13:55:26.875325 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.875205 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" event={"ID":"4d566442-5ddb-47a0-8405-dd2e3d00d3bb","Type":"ContainerDied","Data":"def0020eb47494aaaad0ebfd1b5746afddabb48d99ff4c720369d2ed51db3d36"} Jul 17 13:55:26.875325 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.875262 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" event={"ID":"4d566442-5ddb-47a0-8405-dd2e3d00d3bb","Type":"ContainerStarted","Data":"b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2"} Jul 17 13:55:26.875716 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.875691 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:55:26.877003 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.876983 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerID="4c50800588657d2083f1e9235feaef218fd08f3d7c894f89472f7db2f0df06de" exitCode=0 Jul 17 13:55:26.877095 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.877080 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" event={"ID":"bc38f750-3f04-477b-8ff7-dc84a8c4fafa","Type":"ContainerDied","Data":"4c50800588657d2083f1e9235feaef218fd08f3d7c894f89472f7db2f0df06de"} Jul 17 13:55:26.877159 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.877119 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" event={"ID":"bc38f750-3f04-477b-8ff7-dc84a8c4fafa","Type":"ContainerStarted","Data":"b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90"} Jul 17 13:55:26.887265 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.887240 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd"] Jul 17 13:55:26.891786 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.891763 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:26.905521 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.904914 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd"] Jul 17 13:55:26.936821 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.936775 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8"] Jul 17 13:55:26.938879 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:55:26.938854 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef35ea94_30fe_49c5_bf32_fb71626219e0.slice/crio-36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5 WatchSource:0}: Error finding container 36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5: Status 404 returned error can't find the container with id 36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5 Jul 17 13:55:26.982134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.982110 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:26.982242 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.982181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:26.982242 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:26.982217 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84gpp\" (UniqueName: \"kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.083184 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.083100 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.083184 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.083139 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84gpp\" (UniqueName: \"kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.083362 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.083330 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.083487 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.083465 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.083599 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.083584 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.093005 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.092973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84gpp\" (UniqueName: \"kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.207622 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.207581 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:27.333889 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.333865 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd"] Jul 17 13:55:27.335684 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:55:27.335659 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d43d30_3e5d_4f1d_927c_745d872fd953.slice/crio-416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536 WatchSource:0}: Error finding container 416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536: Status 404 returned error can't find the container with id 416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536 Jul 17 13:55:27.884105 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.884073 2555 generic.go:356] "Generic (PLEG): container finished" podID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerID="0c9d51b16f9dd1c9b5013ee010ab8006d1ef8acfd069611855d8816cc44c6e0e" exitCode=0 Jul 17 13:55:27.884526 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.884157 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" event={"ID":"4d566442-5ddb-47a0-8405-dd2e3d00d3bb","Type":"ContainerDied","Data":"0c9d51b16f9dd1c9b5013ee010ab8006d1ef8acfd069611855d8816cc44c6e0e"} Jul 17 13:55:27.886046 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.886018 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerID="54dd4ac04fd1d33907efb7992a2c172e12637f6019208a4743053a6715f6025e" exitCode=0 Jul 17 13:55:27.886134 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.886105 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" event={"ID":"bc38f750-3f04-477b-8ff7-dc84a8c4fafa","Type":"ContainerDied","Data":"54dd4ac04fd1d33907efb7992a2c172e12637f6019208a4743053a6715f6025e"} Jul 17 13:55:27.887628 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.887604 2555 generic.go:356] "Generic (PLEG): container finished" podID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerID="cbd6c29d9c810ad33dd3a4436cfd4486e800ca1f736e565b605b65ed3674e13e" exitCode=0 Jul 17 13:55:27.887709 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.887692 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" event={"ID":"ef35ea94-30fe-49c5-bf32-fb71626219e0","Type":"ContainerDied","Data":"cbd6c29d9c810ad33dd3a4436cfd4486e800ca1f736e565b605b65ed3674e13e"} Jul 17 13:55:27.887783 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.887741 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" event={"ID":"ef35ea94-30fe-49c5-bf32-fb71626219e0","Type":"ContainerStarted","Data":"36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5"} Jul 17 13:55:27.889321 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.889296 2555 generic.go:356] "Generic (PLEG): container finished" podID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerID="495cf2fb58d310e7f766f1a7964ffc73734642d1f0a0327904fbccddd1dced94" exitCode=0 Jul 17 13:55:27.889424 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.889330 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" event={"ID":"b4d43d30-3e5d-4f1d-927c-745d872fd953","Type":"ContainerDied","Data":"495cf2fb58d310e7f766f1a7964ffc73734642d1f0a0327904fbccddd1dced94"} Jul 17 13:55:27.889424 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:27.889361 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" event={"ID":"b4d43d30-3e5d-4f1d-927c-745d872fd953","Type":"ContainerStarted","Data":"416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536"} Jul 17 13:55:28.895764 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.895719 2555 generic.go:356] "Generic (PLEG): container finished" podID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerID="2d1e251c48fc24b4583ad63cf600546b8e89b932066058c3442fdcc93aac1ccc" exitCode=0 Jul 17 13:55:28.896160 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.895808 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" event={"ID":"b4d43d30-3e5d-4f1d-927c-745d872fd953","Type":"ContainerDied","Data":"2d1e251c48fc24b4583ad63cf600546b8e89b932066058c3442fdcc93aac1ccc"} Jul 17 13:55:28.898051 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.898026 2555 generic.go:356] "Generic (PLEG): container finished" podID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerID="e6b41d037b0188c8d4687e847b0123cd3d17c3b1e31dc13f328ee2c4ba4e85c2" exitCode=0 Jul 17 13:55:28.898164 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.898101 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" event={"ID":"4d566442-5ddb-47a0-8405-dd2e3d00d3bb","Type":"ContainerDied","Data":"e6b41d037b0188c8d4687e847b0123cd3d17c3b1e31dc13f328ee2c4ba4e85c2"} Jul 17 13:55:28.900233 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.900214 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerID="eb35f1eb41548f0e186ff48e961ce198fec10411cd7f26b17c0787375401eb31" exitCode=0 Jul 17 13:55:28.900309 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.900286 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" event={"ID":"bc38f750-3f04-477b-8ff7-dc84a8c4fafa","Type":"ContainerDied","Data":"eb35f1eb41548f0e186ff48e961ce198fec10411cd7f26b17c0787375401eb31"} Jul 17 13:55:28.902046 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.902023 2555 generic.go:356] "Generic (PLEG): container finished" podID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerID="739d8aa2645261e2f1c1d8e2712e7da3368f45b48e1e718488d78eb8d707ae19" exitCode=0 Jul 17 13:55:28.902128 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:28.902106 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" event={"ID":"ef35ea94-30fe-49c5-bf32-fb71626219e0","Type":"ContainerDied","Data":"739d8aa2645261e2f1c1d8e2712e7da3368f45b48e1e718488d78eb8d707ae19"} Jul 17 13:55:29.909584 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:29.909545 2555 generic.go:356] "Generic (PLEG): container finished" podID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerID="5c11c274f53fe7b0f7add1b0432fe280a1f34e27c375d6df0689c4c0e57fc1f5" exitCode=0 Jul 17 13:55:29.910016 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:29.909631 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" event={"ID":"ef35ea94-30fe-49c5-bf32-fb71626219e0","Type":"ContainerDied","Data":"5c11c274f53fe7b0f7add1b0432fe280a1f34e27c375d6df0689c4c0e57fc1f5"} Jul 17 13:55:29.911525 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:29.911500 2555 generic.go:356] "Generic (PLEG): container finished" podID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerID="5f1b523e24d9d594f447dd73fcc6d6295dfea7baf52ded224f3cb601eb5672a0" exitCode=0 Jul 17 13:55:29.911625 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:29.911568 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" event={"ID":"b4d43d30-3e5d-4f1d-927c-745d872fd953","Type":"ContainerDied","Data":"5f1b523e24d9d594f447dd73fcc6d6295dfea7baf52ded224f3cb601eb5672a0"} Jul 17 13:55:30.040139 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.040114 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:30.072963 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.072932 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:30.108565 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108527 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle\") pod \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " Jul 17 13:55:30.108565 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108558 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle\") pod \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " Jul 17 13:55:30.108820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108585 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cfvt\" (UniqueName: \"kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt\") pod \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " Jul 17 13:55:30.108820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108617 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util\") pod \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " Jul 17 13:55:30.108820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108635 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8hbk\" (UniqueName: \"kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk\") pod \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\" (UID: \"4d566442-5ddb-47a0-8405-dd2e3d00d3bb\") " Jul 17 13:55:30.108820 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.108728 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util\") pod \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\" (UID: \"bc38f750-3f04-477b-8ff7-dc84a8c4fafa\") " Jul 17 13:55:30.109307 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.109266 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle" (OuterVolumeSpecName: "bundle") pod "4d566442-5ddb-47a0-8405-dd2e3d00d3bb" (UID: "4d566442-5ddb-47a0-8405-dd2e3d00d3bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:30.109307 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.109289 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle" (OuterVolumeSpecName: "bundle") pod "bc38f750-3f04-477b-8ff7-dc84a8c4fafa" (UID: "bc38f750-3f04-477b-8ff7-dc84a8c4fafa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:30.111404 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.111380 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt" (OuterVolumeSpecName: "kube-api-access-9cfvt") pod "bc38f750-3f04-477b-8ff7-dc84a8c4fafa" (UID: "bc38f750-3f04-477b-8ff7-dc84a8c4fafa"). InnerVolumeSpecName "kube-api-access-9cfvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:55:30.111491 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.111388 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk" (OuterVolumeSpecName: "kube-api-access-h8hbk") pod "4d566442-5ddb-47a0-8405-dd2e3d00d3bb" (UID: "4d566442-5ddb-47a0-8405-dd2e3d00d3bb"). InnerVolumeSpecName "kube-api-access-h8hbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:55:30.114019 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.113996 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util" (OuterVolumeSpecName: "util") pod "bc38f750-3f04-477b-8ff7-dc84a8c4fafa" (UID: "bc38f750-3f04-477b-8ff7-dc84a8c4fafa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:30.114837 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.114818 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util" (OuterVolumeSpecName: "util") pod "4d566442-5ddb-47a0-8405-dd2e3d00d3bb" (UID: "4d566442-5ddb-47a0-8405-dd2e3d00d3bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209869 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209905 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209914 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209922 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cfvt\" (UniqueName: \"kubernetes.io/projected/bc38f750-3f04-477b-8ff7-dc84a8c4fafa-kube-api-access-9cfvt\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209933 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.209958 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.209942 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8hbk\" (UniqueName: \"kubernetes.io/projected/4d566442-5ddb-47a0-8405-dd2e3d00d3bb-kube-api-access-h8hbk\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:30.918311 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.918278 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" Jul 17 13:55:30.918311 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.918309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7" event={"ID":"bc38f750-3f04-477b-8ff7-dc84a8c4fafa","Type":"ContainerDied","Data":"b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90"} Jul 17 13:55:30.918785 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.918359 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b30fa68a86223dcd57934c51ebb676cb99d174b936c7804f2995cae42ccd8d90" Jul 17 13:55:30.920135 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.920106 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" event={"ID":"4d566442-5ddb-47a0-8405-dd2e3d00d3bb","Type":"ContainerDied","Data":"b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2"} Jul 17 13:55:30.920229 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.920156 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5b4650a9cf9d8e53b487cd2f68a524d3f09b4e87550d1c9e3c21eb1b9dba9e2" Jul 17 13:55:30.920780 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.920443 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh" Jul 17 13:55:30.961184 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:30.961154 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:31.024202 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.024166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:31.053358 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.053332 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:31.078794 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.078739 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:31.117191 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117160 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util\") pod \"ef35ea94-30fe-49c5-bf32-fb71626219e0\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " Jul 17 13:55:31.117376 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117226 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle\") pod \"b4d43d30-3e5d-4f1d-927c-745d872fd953\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " Jul 17 13:55:31.117376 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117265 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phtrp\" (UniqueName: \"kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp\") pod \"ef35ea94-30fe-49c5-bf32-fb71626219e0\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " Jul 17 13:55:31.117376 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117296 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle\") pod \"ef35ea94-30fe-49c5-bf32-fb71626219e0\" (UID: \"ef35ea94-30fe-49c5-bf32-fb71626219e0\") " Jul 17 13:55:31.117376 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117325 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84gpp\" (UniqueName: \"kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp\") pod \"b4d43d30-3e5d-4f1d-927c-745d872fd953\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " Jul 17 13:55:31.117586 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117427 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util\") pod \"b4d43d30-3e5d-4f1d-927c-745d872fd953\" (UID: \"b4d43d30-3e5d-4f1d-927c-745d872fd953\") " Jul 17 13:55:31.117786 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.117733 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle" (OuterVolumeSpecName: "bundle") pod "ef35ea94-30fe-49c5-bf32-fb71626219e0" (UID: "ef35ea94-30fe-49c5-bf32-fb71626219e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:31.118078 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.118055 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle" (OuterVolumeSpecName: "bundle") pod "b4d43d30-3e5d-4f1d-927c-745d872fd953" (UID: "b4d43d30-3e5d-4f1d-927c-745d872fd953"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:31.119468 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.119444 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp" (OuterVolumeSpecName: "kube-api-access-phtrp") pod "ef35ea94-30fe-49c5-bf32-fb71626219e0" (UID: "ef35ea94-30fe-49c5-bf32-fb71626219e0"). InnerVolumeSpecName "kube-api-access-phtrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:55:31.119589 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.119572 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp" (OuterVolumeSpecName: "kube-api-access-84gpp") pod "b4d43d30-3e5d-4f1d-927c-745d872fd953" (UID: "b4d43d30-3e5d-4f1d-927c-745d872fd953"). InnerVolumeSpecName "kube-api-access-84gpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:55:31.122950 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.122930 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util" (OuterVolumeSpecName: "util") pod "ef35ea94-30fe-49c5-bf32-fb71626219e0" (UID: "ef35ea94-30fe-49c5-bf32-fb71626219e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:31.123623 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.123571 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util" (OuterVolumeSpecName: "util") pod "b4d43d30-3e5d-4f1d-927c-745d872fd953" (UID: "b4d43d30-3e5d-4f1d-927c-745d872fd953"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.217947 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.217984 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.217995 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phtrp\" (UniqueName: \"kubernetes.io/projected/ef35ea94-30fe-49c5-bf32-fb71626219e0-kube-api-access-phtrp\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.218006 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef35ea94-30fe-49c5-bf32-fb71626219e0-bundle\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.218015 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84gpp\" (UniqueName: \"kubernetes.io/projected/b4d43d30-3e5d-4f1d-927c-745d872fd953-kube-api-access-84gpp\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.218041 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.218023 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4d43d30-3e5d-4f1d-927c-745d872fd953-util\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:55:31.926423 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.926390 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" Jul 17 13:55:31.926423 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.926409 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8" event={"ID":"ef35ea94-30fe-49c5-bf32-fb71626219e0","Type":"ContainerDied","Data":"36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5"} Jul 17 13:55:31.926854 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.926456 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36cc7fc9ff14a58b1c693131d327da67e9d74200697d12ab6da7b6bfb0f228d5" Jul 17 13:55:31.928343 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.928319 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" event={"ID":"b4d43d30-3e5d-4f1d-927c-745d872fd953","Type":"ContainerDied","Data":"416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536"} Jul 17 13:55:31.928463 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.928357 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd" Jul 17 13:55:31.928463 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:31.928358 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="416333365eb8d4d1062bace19b5edb2ac08fa93810e295cbdafd09e317d37536" Jul 17 13:55:33.044323 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:33.044289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:33.638396 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:33.638355 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:34.688406 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:34.688376 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:34.786937 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:34.786898 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:35.582063 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:35.581983 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:35.683534 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:35.683504 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:37.899478 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:37.899439 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:40.655177 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:40.655143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:40.914818 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:40.914728 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:53.338287 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:53.338252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:53.434789 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:53.434727 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:55.042722 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042683 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5"] Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042936 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042947 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042956 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042962 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042971 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042977 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042983 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042989 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.042995 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043000 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043006 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043010 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043017 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043022 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043029 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043033 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043041 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043048 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043053 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043058 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="extract" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043064 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043069 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="pull" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043075 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="util" Jul 17 13:55:55.043099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043080 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="util" Jul 17 13:55:55.043815 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043123 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d566442-5ddb-47a0-8405-dd2e3d00d3bb" containerName="extract" Jul 17 13:55:55.043815 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043129 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef35ea94-30fe-49c5-bf32-fb71626219e0" containerName="extract" Jul 17 13:55:55.043815 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043135 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d43d30-3e5d-4f1d-927c-745d872fd953" containerName="extract" Jul 17 13:55:55.043815 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.043140 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc38f750-3f04-477b-8ff7-dc84a8c4fafa" containerName="extract" Jul 17 13:55:55.045840 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.045823 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.048026 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.048003 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 13:55:55.048128 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.048084 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 13:55:55.048440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.048422 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 13:55:55.048899 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.048883 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 13:55:55.048995 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.048919 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gvwvs\"" Jul 17 13:55:55.055108 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.055088 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5"] Jul 17 13:55:55.057960 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.057936 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:55.097393 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.097352 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.097393 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.097394 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndvt4\" (UniqueName: \"kubernetes.io/projected/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-kube-api-access-ndvt4\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.097596 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.097426 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.198237 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.198204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.198389 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.198245 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndvt4\" (UniqueName: \"kubernetes.io/projected/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-kube-api-access-ndvt4\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.198389 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.198273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.198903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.198876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.200695 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.200677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.206247 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.206225 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndvt4\" (UniqueName: \"kubernetes.io/projected/a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a-kube-api-access-ndvt4\") pod \"kuadrant-console-plugin-66b5774985-jtzj5\" (UID: \"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.242268 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.242223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:55.355232 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.355146 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" Jul 17 13:55:55.483180 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:55.483146 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5"] Jul 17 13:55:55.486206 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:55:55.486177 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda20566d9_8bb8_4ccc_9810_aeb5a8e9fe1a.slice/crio-6dd3c4dc6cbd6f3a24ea78ec73b17472e795eb0799c33206694b143c47847a34 WatchSource:0}: Error finding container 6dd3c4dc6cbd6f3a24ea78ec73b17472e795eb0799c33206694b143c47847a34: Status 404 returned error can't find the container with id 6dd3c4dc6cbd6f3a24ea78ec73b17472e795eb0799c33206694b143c47847a34 Jul 17 13:55:56.031976 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:56.031940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" event={"ID":"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a","Type":"ContainerStarted","Data":"6dd3c4dc6cbd6f3a24ea78ec73b17472e795eb0799c33206694b143c47847a34"} Jul 17 13:55:56.268718 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:56.268671 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:56.367782 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:56.367685 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:58.566450 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:58.566420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:58.765919 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:58.765885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:58.864369 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:58.864278 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:59.708361 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:59.708315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:55:59.808563 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:55:59.808520 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:07.765264 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:07.765226 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:07.863233 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:07.863198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:08.410831 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:08.410796 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:08.515550 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:08.515506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:11.363455 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:11.363415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:11.463923 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:11.463892 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:15.958480 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:15.958446 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:16.059440 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:16.059404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:18.151946 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:18.151901 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" event={"ID":"a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a","Type":"ContainerStarted","Data":"409b494049b0d90579cd45bde57ea3f19768e218d96658c8bc72a81ba9b4ebf6"} Jul 17 13:56:18.185546 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:18.185485 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jtzj5" podStartSLOduration=1.227517303 podStartE2EDuration="23.18546327s" totalImagesPullingTime="21.957945967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:18.179911274 +0000 UTC m=+664.940776287" watchObservedRunningTime="2026-07-17 13:56:18.18546327 +0000 UTC m=+664.946328283" Jul 17 13:56:18.715439 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:18.715398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:18.822343 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:18.822311 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:30.042454 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:30.042415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:30.142658 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:30.142626 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:38.272075 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:38.272040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:38.608400 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:38.608321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:50.509799 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.509740 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:50.614385 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.614353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:50.679850 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.679821 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:50.927704 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.927629 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:56:50.952066 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.952037 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:56:50.952218 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.952156 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:50.954878 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.954859 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 17 13:56:50.960456 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:50.960433 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:56:51.023779 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.023710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jhv\" (UniqueName: \"kubernetes.io/projected/7e571386-1ac9-4dbd-b949-76f56f66713d-kube-api-access-c7jhv\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.023779 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.023787 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e571386-1ac9-4dbd-b949-76f56f66713d-config-file\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.124308 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.124273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jhv\" (UniqueName: \"kubernetes.io/projected/7e571386-1ac9-4dbd-b949-76f56f66713d-kube-api-access-c7jhv\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.124308 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.124315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e571386-1ac9-4dbd-b949-76f56f66713d-config-file\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.124954 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.124936 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e571386-1ac9-4dbd-b949-76f56f66713d-config-file\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.135474 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.135451 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jhv\" (UniqueName: \"kubernetes.io/projected/7e571386-1ac9-4dbd-b949-76f56f66713d-kube-api-access-c7jhv\") pod \"limitador-limitador-54d6d988d5-f4hw7\" (UID: \"7e571386-1ac9-4dbd-b949-76f56f66713d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.262121 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.262083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:51.395132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.395107 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:51.395521 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.395503 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:56:51.397099 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:56:51.397072 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e571386_1ac9_4dbd_b949_76f56f66713d.slice/crio-6541284451e1a74a49b0d79d6736cb4b153dbf039747e7abd8e233e26390bd7e WatchSource:0}: Error finding container 6541284451e1a74a49b0d79d6736cb4b153dbf039747e7abd8e233e26390bd7e: Status 404 returned error can't find the container with id 6541284451e1a74a49b0d79d6736cb4b153dbf039747e7abd8e233e26390bd7e Jul 17 13:56:51.494323 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.494290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:56:51.573600 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.573517 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:56:51.577787 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.577746 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:56:51.579947 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.579925 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-km8n8\"" Jul 17 13:56:51.585934 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.585913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:56:51.627978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.627941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdll\" (UniqueName: \"kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll\") pod \"authorino-75d556bd77-4l6v5\" (UID: \"dcbf296d-59cc-411e-9ac6-dbd55a164f09\") " pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:56:51.728794 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.728736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdll\" (UniqueName: \"kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll\") pod \"authorino-75d556bd77-4l6v5\" (UID: \"dcbf296d-59cc-411e-9ac6-dbd55a164f09\") " pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:56:51.737439 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.737383 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdll\" (UniqueName: \"kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll\") pod \"authorino-75d556bd77-4l6v5\" (UID: \"dcbf296d-59cc-411e-9ac6-dbd55a164f09\") " pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:56:51.887538 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:51.887338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:56:52.070328 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:52.070274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:56:52.073881 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:56:52.073852 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcbf296d_59cc_411e_9ac6_dbd55a164f09.slice/crio-88ce402996a6d00a7288ce61f09aa8eebd7b3979ac7114fb0ec01d418b3271e4 WatchSource:0}: Error finding container 88ce402996a6d00a7288ce61f09aa8eebd7b3979ac7114fb0ec01d418b3271e4: Status 404 returned error can't find the container with id 88ce402996a6d00a7288ce61f09aa8eebd7b3979ac7114fb0ec01d418b3271e4 Jul 17 13:56:52.312065 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:52.312026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" event={"ID":"7e571386-1ac9-4dbd-b949-76f56f66713d","Type":"ContainerStarted","Data":"6541284451e1a74a49b0d79d6736cb4b153dbf039747e7abd8e233e26390bd7e"} Jul 17 13:56:52.313388 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:52.313362 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4l6v5" event={"ID":"dcbf296d-59cc-411e-9ac6-dbd55a164f09","Type":"ContainerStarted","Data":"88ce402996a6d00a7288ce61f09aa8eebd7b3979ac7114fb0ec01d418b3271e4"} Jul 17 13:56:54.326598 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:54.326558 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" event={"ID":"7e571386-1ac9-4dbd-b949-76f56f66713d","Type":"ContainerStarted","Data":"6fd681ccde961f9bdd3da32fa063ee2e3f44ea2794a4768b975a5dad0971016f"} Jul 17 13:56:54.327127 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:54.326705 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:56:54.346740 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:54.346679 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" podStartSLOduration=1.842404959 podStartE2EDuration="4.346657222s" totalImagesPullingTime="2.504252263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:54.344715223 +0000 UTC m=+701.105580236" watchObservedRunningTime="2026-07-17 13:56:54.346657222 +0000 UTC m=+701.107522236" Jul 17 13:56:56.339192 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:56.339153 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4l6v5" event={"ID":"dcbf296d-59cc-411e-9ac6-dbd55a164f09","Type":"ContainerStarted","Data":"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf"} Jul 17 13:56:56.357285 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:56:56.357215 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4l6v5" podStartSLOduration=1.244064186 podStartE2EDuration="5.357192872s" totalImagesPullingTime="4.113128686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:56.356729043 +0000 UTC m=+703.117594058" watchObservedRunningTime="2026-07-17 13:56:56.357192872 +0000 UTC m=+703.118057887" Jul 17 13:57:05.332367 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:05.332288 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f4hw7" Jul 17 13:57:07.042765 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:07.042727 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:07.153864 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:07.153830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:07.515353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:07.515318 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:10.229176 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:10.229145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:10.289903 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:10.289869 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:20.174310 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.174265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:20.677940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.677906 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:20.738737 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.738704 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:20.742271 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.742253 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:20.752972 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.752945 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:20.857380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.857338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb8b5\" (UniqueName: \"kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5\") pod \"authorino-549ff686cd-mzg75\" (UID: \"9f72718e-a359-4569-867b-c6a0390d86cf\") " pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:20.958263 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.958167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb8b5\" (UniqueName: \"kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5\") pod \"authorino-549ff686cd-mzg75\" (UID: \"9f72718e-a359-4569-867b-c6a0390d86cf\") " pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:20.967283 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:20.967251 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb8b5\" (UniqueName: \"kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5\") pod \"authorino-549ff686cd-mzg75\" (UID: \"9f72718e-a359-4569-867b-c6a0390d86cf\") " pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:21.050705 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:21.050672 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:21.053040 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:21.053015 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:21.201436 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:21.201372 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:21.205017 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:57:21.204986 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f72718e_a359_4569_867b_c6a0390d86cf.slice/crio-69ec93bd844efe69e70ecd0ce1969b644e721d45d48b44db3ae049767bc3d92c WatchSource:0}: Error finding container 69ec93bd844efe69e70ecd0ce1969b644e721d45d48b44db3ae049767bc3d92c: Status 404 returned error can't find the container with id 69ec93bd844efe69e70ecd0ce1969b644e721d45d48b44db3ae049767bc3d92c Jul 17 13:57:21.459413 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:21.459370 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-549ff686cd-mzg75" event={"ID":"9f72718e-a359-4569-867b-c6a0390d86cf","Type":"ContainerStarted","Data":"69ec93bd844efe69e70ecd0ce1969b644e721d45d48b44db3ae049767bc3d92c"} Jul 17 13:57:22.465279 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.465243 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-549ff686cd-mzg75" event={"ID":"9f72718e-a359-4569-867b-c6a0390d86cf","Type":"ContainerStarted","Data":"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109"} Jul 17 13:57:22.465703 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.465316 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-549ff686cd-mzg75" podUID="9f72718e-a359-4569-867b-c6a0390d86cf" containerName="authorino" containerID="cri-o://bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109" gracePeriod=30 Jul 17 13:57:22.481676 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.481628 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-549ff686cd-mzg75" podStartSLOduration=2.103467428 podStartE2EDuration="2.481612131s" totalImagesPullingTime="378.144703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:57:22.481218053 +0000 UTC m=+729.242083066" watchObservedRunningTime="2026-07-17 13:57:22.481612131 +0000 UTC m=+729.242477176" Jul 17 13:57:22.715648 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.715585 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:22.770978 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.770943 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb8b5\" (UniqueName: \"kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5\") pod \"9f72718e-a359-4569-867b-c6a0390d86cf\" (UID: \"9f72718e-a359-4569-867b-c6a0390d86cf\") " Jul 17 13:57:22.773183 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.773150 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5" (OuterVolumeSpecName: "kube-api-access-xb8b5") pod "9f72718e-a359-4569-867b-c6a0390d86cf" (UID: "9f72718e-a359-4569-867b-c6a0390d86cf"). InnerVolumeSpecName "kube-api-access-xb8b5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:57:22.872672 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.872636 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb8b5\" (UniqueName: \"kubernetes.io/projected/9f72718e-a359-4569-867b-c6a0390d86cf-kube-api-access-xb8b5\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:57:22.920130 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.920093 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:57:22.920291 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:22.920263 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4l6v5" podUID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" containerName="authorino" containerID="cri-o://f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf" gracePeriod=30 Jul 17 13:57:23.154005 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.153980 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:57:23.275472 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.275439 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmdll\" (UniqueName: \"kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll\") pod \"dcbf296d-59cc-411e-9ac6-dbd55a164f09\" (UID: \"dcbf296d-59cc-411e-9ac6-dbd55a164f09\") " Jul 17 13:57:23.277517 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.277495 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll" (OuterVolumeSpecName: "kube-api-access-wmdll") pod "dcbf296d-59cc-411e-9ac6-dbd55a164f09" (UID: "dcbf296d-59cc-411e-9ac6-dbd55a164f09"). InnerVolumeSpecName "kube-api-access-wmdll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:57:23.376052 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.375965 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmdll\" (UniqueName: \"kubernetes.io/projected/dcbf296d-59cc-411e-9ac6-dbd55a164f09-kube-api-access-wmdll\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 13:57:23.471309 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.471276 2555 generic.go:356] "Generic (PLEG): container finished" podID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" containerID="f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf" exitCode=0 Jul 17 13:57:23.471805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.471316 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4l6v5" Jul 17 13:57:23.471805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.471354 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4l6v5" event={"ID":"dcbf296d-59cc-411e-9ac6-dbd55a164f09","Type":"ContainerDied","Data":"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf"} Jul 17 13:57:23.471805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.471388 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4l6v5" event={"ID":"dcbf296d-59cc-411e-9ac6-dbd55a164f09","Type":"ContainerDied","Data":"88ce402996a6d00a7288ce61f09aa8eebd7b3979ac7114fb0ec01d418b3271e4"} Jul 17 13:57:23.471805 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.471408 2555 scope.go:117] "RemoveContainer" containerID="f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf" Jul 17 13:57:23.472662 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.472590 2555 generic.go:356] "Generic (PLEG): container finished" podID="9f72718e-a359-4569-867b-c6a0390d86cf" containerID="bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109" exitCode=0 Jul 17 13:57:23.472662 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.472632 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-549ff686cd-mzg75" Jul 17 13:57:23.472799 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.472636 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-549ff686cd-mzg75" event={"ID":"9f72718e-a359-4569-867b-c6a0390d86cf","Type":"ContainerDied","Data":"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109"} Jul 17 13:57:23.472799 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.472725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-549ff686cd-mzg75" event={"ID":"9f72718e-a359-4569-867b-c6a0390d86cf","Type":"ContainerDied","Data":"69ec93bd844efe69e70ecd0ce1969b644e721d45d48b44db3ae049767bc3d92c"} Jul 17 13:57:23.485933 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.485916 2555 scope.go:117] "RemoveContainer" containerID="f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf" Jul 17 13:57:23.486243 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:57:23.486226 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf\": container with ID starting with f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf not found: ID does not exist" containerID="f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf" Jul 17 13:57:23.486281 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.486253 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf"} err="failed to get container status \"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf\": rpc error: code = NotFound desc = could not find container \"f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf\": container with ID starting with f4df92cd7e94af30b973b7b9ac12c76f46ee2b230b088b3703405fda41fcf2bf not found: ID does not exist" Jul 17 13:57:23.486281 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.486271 2555 scope.go:117] "RemoveContainer" containerID="bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109" Jul 17 13:57:23.494482 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.494463 2555 scope.go:117] "RemoveContainer" containerID="bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109" Jul 17 13:57:23.494762 ip-10-0-133-123 kubenswrapper[2555]: E0717 13:57:23.494728 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109\": container with ID starting with bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109 not found: ID does not exist" containerID="bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109" Jul 17 13:57:23.494811 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.494770 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109"} err="failed to get container status \"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109\": rpc error: code = NotFound desc = could not find container \"bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109\": container with ID starting with bc5a9f18c228cc7f498ff12a6764429cca735bb40b9e088b627475892772a109 not found: ID does not exist" Jul 17 13:57:23.498077 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.498056 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:57:23.501515 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.501494 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4l6v5"] Jul 17 13:57:23.511727 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.511700 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:23.516230 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.516211 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-549ff686cd-mzg75"] Jul 17 13:57:23.678465 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.678393 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f72718e-a359-4569-867b-c6a0390d86cf" path="/var/lib/kubelet/pods/9f72718e-a359-4569-867b-c6a0390d86cf/volumes" Jul 17 13:57:23.678698 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:23.678685 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" path="/var/lib/kubelet/pods/dcbf296d-59cc-411e-9ac6-dbd55a164f09/volumes" Jul 17 13:57:27.532198 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:27.532157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:27.630036 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:27.630003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:27.714099 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:27.714065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:27.821045 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:27.820969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:30.943827 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:30.943792 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:57:31.132855 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:57:31.132822 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:03.787520 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:03.787476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:03.885535 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:03.885501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:04.445691 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:04.445661 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:04.584303 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:04.584272 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:04.956264 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:04.956235 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:05.056243 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:05.056207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:05.195592 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:05.195561 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:58:14.139720 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:14.139683 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:14.286055 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:14.286016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:32.210238 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:32.210201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:32.681722 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:32.681688 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:58:45.794246 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:45.794159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:58:46.581177 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:58:46.581141 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:59:01.298428 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:01.298384 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:59:02.576165 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:02.576124 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:59:15.592111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592072 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g"] Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592374 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f72718e-a359-4569-867b-c6a0390d86cf" containerName="authorino" Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592386 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f72718e-a359-4569-867b-c6a0390d86cf" containerName="authorino" Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592396 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" containerName="authorino" Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592402 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" containerName="authorino" Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592467 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbf296d-59cc-411e-9ac6-dbd55a164f09" containerName="authorino" Jul 17 13:59:15.592539 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.592477 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f72718e-a359-4569-867b-c6a0390d86cf" containerName="authorino" Jul 17 13:59:15.595256 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.595241 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.598371 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.598342 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 17 13:59:15.598371 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.598362 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wn44r\"" Jul 17 13:59:15.598606 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.598352 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 13:59:15.598606 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.598352 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 13:59:15.605635 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.605613 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g"] Jul 17 13:59:15.618043 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.618015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:59:15.735158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.735158 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735164 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.735372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735235 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2skdr\" (UniqueName: \"kubernetes.io/projected/420b1987-3a97-4d23-9405-6d32044e2675-kube-api-access-2skdr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.735372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735272 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.735372 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735354 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420b1987-3a97-4d23-9405-6d32044e2675-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.735467 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.735379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836080 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836085 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836101 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836237 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skdr\" (UniqueName: \"kubernetes.io/projected/420b1987-3a97-4d23-9405-6d32044e2675-kube-api-access-2skdr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836353 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836289 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836525 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836373 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420b1987-3a97-4d23-9405-6d32044e2675-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836525 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836459 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836525 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836514 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.836679 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.836614 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.838531 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.838503 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420b1987-3a97-4d23-9405-6d32044e2675-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.838781 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.838763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420b1987-3a97-4d23-9405-6d32044e2675-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.845171 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.845097 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2skdr\" (UniqueName: \"kubernetes.io/projected/420b1987-3a97-4d23-9405-6d32044e2675-kube-api-access-2skdr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g\" (UID: \"420b1987-3a97-4d23-9405-6d32044e2675\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:15.904674 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:15.904632 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:16.037528 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:16.037500 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g"] Jul 17 13:59:16.040363 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:59:16.040336 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420b1987_3a97_4d23_9405_6d32044e2675.slice/crio-e7c8a049dd62133a29722e4351175e1f644913abfdbfb3e3995fde378a85ef0a WatchSource:0}: Error finding container e7c8a049dd62133a29722e4351175e1f644913abfdbfb3e3995fde378a85ef0a: Status 404 returned error can't find the container with id e7c8a049dd62133a29722e4351175e1f644913abfdbfb3e3995fde378a85ef0a Jul 17 13:59:16.989011 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:16.988968 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" event={"ID":"420b1987-3a97-4d23-9405-6d32044e2675","Type":"ContainerStarted","Data":"e7c8a049dd62133a29722e4351175e1f644913abfdbfb3e3995fde378a85ef0a"} Jul 17 13:59:17.479180 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:17.479095 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:59:17.995592 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:17.995555 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" event={"ID":"420b1987-3a97-4d23-9405-6d32044e2675","Type":"ContainerStarted","Data":"80e3fe8cbad5685876146407cde031f4444bc37cfea22baa6f96323d8f92fbf5"} Jul 17 13:59:18.016771 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:18.016698 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" podStartSLOduration=1.8844646950000001 podStartE2EDuration="3.016677627s" totalImagesPullingTime="1.132212932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:59:18.016448799 +0000 UTC m=+844.777313809" watchObservedRunningTime="2026-07-17 13:59:18.016677627 +0000 UTC m=+844.777542640" Jul 17 13:59:25.471203 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.471172 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj"] Jul 17 13:59:25.474617 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.474599 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.476971 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.476947 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 17 13:59:25.482780 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.482740 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj"] Jul 17 13:59:25.495674 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.495641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:59:25.610694 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610645 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.610893 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9r75\" (UniqueName: \"kubernetes.io/projected/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-kube-api-access-w9r75\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.610893 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610740 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.610893 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610797 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.610893 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610816 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.610893 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.610858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712087 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9r75\" (UniqueName: \"kubernetes.io/projected/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-kube-api-access-w9r75\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712132 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712133 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712171 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712215 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712380 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712244 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712664 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712637 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712664 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712653 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.712802 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.712708 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.714547 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.714528 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.714693 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.714677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.720662 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.720643 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9r75\" (UniqueName: \"kubernetes.io/projected/ff0487e9-50bd-4e2e-8485-8414cc3e9a2b-kube-api-access-w9r75\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swhrj\" (UID: \"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.785720 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.785678 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:25.905091 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.905059 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:25.905252 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.905113 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:25.914463 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.914443 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj"] Jul 17 13:59:25.916375 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:59:25.916340 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0487e9_50bd_4e2e_8485_8414cc3e9a2b.slice/crio-dd14c26ebca260261554425394b67433531e165be2b8dbca0a27c6961832335f WatchSource:0}: Error finding container dd14c26ebca260261554425394b67433531e165be2b8dbca0a27c6961832335f: Status 404 returned error can't find the container with id dd14c26ebca260261554425394b67433531e165be2b8dbca0a27c6961832335f Jul 17 13:59:25.918079 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:25.918054 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:26.033085 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:26.033048 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" event={"ID":"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b","Type":"ContainerStarted","Data":"dd14c26ebca260261554425394b67433531e165be2b8dbca0a27c6961832335f"} Jul 17 13:59:26.044379 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:26.044313 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g" Jul 17 13:59:27.039021 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:27.038980 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" event={"ID":"ff0487e9-50bd-4e2e-8485-8414cc3e9a2b","Type":"ContainerStarted","Data":"be17a4459eac619b33c4097690f0ae42647696fa3189ca9510d545d237d2ff8a"} Jul 17 13:59:27.057568 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:27.057520 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" podStartSLOduration=1.813402973 podStartE2EDuration="2.057503243s" totalImagesPullingTime="244.10027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:59:27.05562196 +0000 UTC m=+853.816486973" watchObservedRunningTime="2026-07-17 13:59:27.057503243 +0000 UTC m=+853.818368255" Jul 17 13:59:27.579633 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:27.579592 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:59:35.774722 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.774686 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp"] Jul 17 13:59:35.778398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.778376 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.780918 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.780900 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 17 13:59:35.783830 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.783808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.783940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.783844 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m6tg\" (UniqueName: \"kubernetes.io/projected/eaff2e3d-a06d-48ba-8750-f0f099aa8098-kube-api-access-8m6tg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.783940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.783875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.783940 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.783933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.784111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.783974 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.784111 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.784018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.786419 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.786397 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:35.786512 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.786456 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:35.789398 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.789377 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp"] Jul 17 13:59:35.800181 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.800162 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:35.823848 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.823809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 13:59:35.884488 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884450 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884533 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m6tg\" (UniqueName: \"kubernetes.io/projected/eaff2e3d-a06d-48ba-8750-f0f099aa8098-kube-api-access-8m6tg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884678 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884886 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884720 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884886 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884792 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.884987 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.884956 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.885122 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.885104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.885219 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.885199 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.886819 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.886797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaff2e3d-a06d-48ba-8750-f0f099aa8098-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.887186 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.887164 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaff2e3d-a06d-48ba-8750-f0f099aa8098-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:35.892490 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:35.892467 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m6tg\" (UniqueName: \"kubernetes.io/projected/eaff2e3d-a06d-48ba-8750-f0f099aa8098-kube-api-access-8m6tg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp\" (UID: \"eaff2e3d-a06d-48ba-8750-f0f099aa8098\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:36.088741 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:36.088657 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:36.088897 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:36.088811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swhrj" Jul 17 13:59:36.230141 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:36.230110 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp"] Jul 17 13:59:36.233323 ip-10-0-133-123 kubenswrapper[2555]: W0717 13:59:36.233296 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeaff2e3d_a06d_48ba_8750_f0f099aa8098.slice/crio-3d17b020513643fc564547a60c74681b564826aabf5852c5003217b76c322102 WatchSource:0}: Error finding container 3d17b020513643fc564547a60c74681b564826aabf5852c5003217b76c322102: Status 404 returned error can't find the container with id 3d17b020513643fc564547a60c74681b564826aabf5852c5003217b76c322102 Jul 17 13:59:37.083485 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:37.083441 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" event={"ID":"eaff2e3d-a06d-48ba-8750-f0f099aa8098","Type":"ContainerStarted","Data":"514712feaed0d59439b64534efddca1f79f0c6c29681fd8752bf6ffb26f9f4bd"} Jul 17 13:59:37.084180 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:37.083494 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" event={"ID":"eaff2e3d-a06d-48ba-8750-f0f099aa8098","Type":"ContainerStarted","Data":"3d17b020513643fc564547a60c74681b564826aabf5852c5003217b76c322102"} Jul 17 13:59:37.100279 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:37.100226 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" podStartSLOduration=1.9111556680000001 podStartE2EDuration="2.100206273s" totalImagesPullingTime="189.050605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:59:37.099559439 +0000 UTC m=+863.860424453" watchObservedRunningTime="2026-07-17 13:59:37.100206273 +0000 UTC m=+863.861071287" Jul 17 13:59:39.783726 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:39.783679 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 13:59:46.089015 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:46.088973 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:46.089451 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:46.089034 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:46.101336 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:46.101305 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 13:59:46.136561 ip-10-0-133-123 kubenswrapper[2555]: I0717 13:59:46.136531 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp" Jul 17 14:00:19.913016 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:00:19.912975 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:00:21.245136 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:00:21.245106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:00:48.302373 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:00:48.302333 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:00:58.678774 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:00:58.678714 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:01:06.091412 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:06.091365 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:01:09.484711 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:09.484676 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:01:14.096170 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:14.096125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:01:29.804050 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:29.804014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:01:43.105438 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:43.105354 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:01:43.223878 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:01:43.223843 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:02:25.860119 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:02:25.860084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:02:27.108448 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:02:27.108413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:02:49.678210 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:02:49.678180 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:03:00.992031 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:00.991995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:01.091392 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:01.091336 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:01.224912 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:01.224879 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:07.289324 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:07.289246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:07.366923 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:07.366887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:07.469246 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:07.469212 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:07.560297 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:07.560217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:07.820388 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:07.820307 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:13.462239 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:13.462201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:13.526605 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:13.526570 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:29.481954 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:29.481920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:03:36.990302 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:36.990269 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:03:40.483277 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:40.483238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:03:45.886360 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:45.886326 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:03:54.380064 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:03:54.380024 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:04:05.218438 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:04:05.218400 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:04:15.386886 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:04:15.386843 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:04:27.287156 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:04:27.287122 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:04:36.880097 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:04:36.880006 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:05:02.688445 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:05:02.688400 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:05:12.581236 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:05:12.581199 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:05:35.679515 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:05:35.679477 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:05:46.482676 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:05:46.482638 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:06:01.985423 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:06:01.985386 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:06:12.393194 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:06:12.393103 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:06:30.483961 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:06:30.483925 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:06:40.485662 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:06:40.485627 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:07:09.668818 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:07:09.668731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:07:11.015585 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:07:11.015552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:08:33.405091 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:08:33.405051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:08:46.982698 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:08:46.982661 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:08:56.629828 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:08:56.629790 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:08:56.649124 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:08:56.649089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:09:08.618057 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:09:08.617956 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:09:19.481240 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:09:19.481199 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:09:29.889092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:09:29.889056 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:09:39.187817 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:09:39.187782 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:09:50.685885 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:09:50.685840 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:10:01.397907 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:10:01.397870 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:10:09.987390 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:10:09.987357 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:10:23.088614 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:10:23.088575 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:10:42.188767 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:10:42.188728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:10:52.090734 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:10:52.090697 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:01.786985 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:01.786952 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:08.029742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:08.029710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:11:12.486636 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:12.486597 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:29.386370 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:29.386335 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:40.386700 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:40.386654 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:49.582391 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:49.582353 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:11:58.683106 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:58.683070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:11:59.989850 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:11:59.989811 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:12:10.188962 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:12:10.188867 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:12:20.988805 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:12:20.988766 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:12:37.586217 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:12:37.586179 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:12:51.684535 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:12:51.684502 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:04.085205 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:04.085174 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:15.185108 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:15.185072 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:25.884879 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:25.884846 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:36.777640 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:36.777550 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:47.490097 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:47.490064 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:13:56.884643 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:13:56.884608 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:14:05.088534 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:05.088499 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:14:09.618729 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:09.618685 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:11.028468 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:11.028429 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:16.011660 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:16.011620 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:16.081985 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:16.081937 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:16.235053 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:16.235015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:16.308673 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:16.308577 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:17.356804 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:17.356773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:17.458849 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:17.458813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:17.558319 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:17.558277 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:17.816952 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:17.816912 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:18.115557 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:18.115472 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:18.213948 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:18.213914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:20.165461 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:20.165417 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:14:20.294354 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:14:20.294321 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:16:29.737029 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:16:29.736988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:16:32.488635 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:16:32.488601 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:22.541349 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:22.541307 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:22.645741 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:22.645706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:25.315767 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:25.315716 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:17:29.737033 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:29.736989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:30.247413 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:30.247374 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:35.448251 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:35.448190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:35.526189 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:35.526147 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:35.711451 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:35.711361 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:35.814556 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:35.814523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:35.922704 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:35.922670 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:37.153868 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:37.153834 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:37.366450 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:37.366411 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:37.563904 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:37.563866 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:37.912398 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:37.912305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:38.013535 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:38.013496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:39.591073 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:39.591035 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:17:44.144740 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:44.144708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:44.202937 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:44.202893 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:47.722121 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:47.722080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:48.884513 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:48.884425 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:48.983852 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:48.983809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:59.710874 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:59.710818 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:17:59.812257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:17:59.812220 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:38.729376 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:38.729338 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:38.830896 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:38.830862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:41.812037 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:41.812004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:41.912871 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:41.912831 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:44.491283 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:44.491232 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:18:47.254060 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:47.254023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:48.263060 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:48.263029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:48.385333 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:48.385293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:54.304960 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:54.304917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:54.407448 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:54.407416 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:54.545036 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:54.545003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:54.644405 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:54.644308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:55.652351 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:55.652274 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:55.748260 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:55.748222 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:55.886981 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:55.886945 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:56.053950 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:56.053914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:56.353959 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:56.353854 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:56.453624 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:56.453573 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:58.640695 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:58.640644 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:18:59.290949 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:18:59.290913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:22:49.720778 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:22:49.720675 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:22:51.252271 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:22:51.252204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:23:02.480061 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:23:02.480027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:23:02.579234 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:23:02.579203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:23:12.321496 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:23:12.321459 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:23:41.051676 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:23:41.051640 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:24:19.880176 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:24:19.880075 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:01.319150 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:01.319110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:01.422416 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:01.422390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:04.183134 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:04.183087 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:25:06.663037 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:06.662997 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:07.610443 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:07.610398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:12.742668 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:12.742625 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:12.945765 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:12.945715 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:13.043409 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:13.043376 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:13.209500 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:13.209466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:13.295561 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:13.295483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:14.342109 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:14.342071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:14.437829 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:14.437801 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:14.542176 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:14.542134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:14.855613 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:14.855576 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:15.001878 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:15.001845 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:15.097167 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:15.097133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:16.787740 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:16.787708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:25:18.183902 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:25:18.183858 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:27:50.786383 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:27:50.786345 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:27:50.874640 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:27:50.874606 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:27:59.517480 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:27:59.517403 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:00.628945 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:00.628913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:18.941192 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:18.941153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:19.039630 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:19.039596 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:21.781802 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:21.781768 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:28:24.237396 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:24.237357 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:25.564402 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:25.564356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:31.686085 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:31.686048 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:31.760017 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:31.759982 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:32.196430 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:32.196380 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:32.296313 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:32.296275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:32.458648 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:32.458534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:33.924597 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:33.924538 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:40.181423 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:40.181337 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:40.284890 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:40.284851 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:40.398000 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:40.397967 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:40.478311 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:40.478227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:40.906288 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:40.906250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:28:42.037491 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:28:42.037446 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:06.881309 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:06.881272 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:14.513205 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:14.513164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:14.618840 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:14.618806 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:14.712159 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:14.712126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:30.639982 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:30.639946 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:36.981067 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:36.981032 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:37.091335 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:37.091300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:37.185870 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:37.185841 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:40.980270 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:40.980233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:47.714413 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:47.714367 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:48.521673 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:48.521636 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:55.917328 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:55.917294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:56.097407 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:56.097370 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:56.188958 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:56.188883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:57.014479 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:57.014433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:57.115963 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:57.115924 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:58.858239 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:58.858207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:29:59.486529 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:29:59.486498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:30:05.066966 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:05.066933 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:05.147288 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:05.147256 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:05.746169 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:05.746124 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:06.998327 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:06.998294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:07.063113 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:07.063078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:11.872482 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:11.872442 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:11.971582 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:11.971548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:12.146162 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:12.146080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:12.572206 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:12.572169 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:30:12.622496 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:12.622456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:12.989464 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:12.989433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:15.997099 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:15.997060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:16.098260 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:16.098223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:16.197086 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:16.197050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:16.712971 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:16.712932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:16.943422 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:16.943383 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:17.159920 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:17.159887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:20.506434 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:20.506402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:21.237363 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:21.237329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:21.332449 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:21.332419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:21.626356 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:21.626322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:27.701110 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:27.701075 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:30:28.326937 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:28.326906 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:28.435104 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:28.435069 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:39.258046 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:39.257991 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:42.295100 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:42.295064 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:30:43.780413 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:43.780361 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:44.197712 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:44.197604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:50.549564 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:50.549501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:50.623049 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:50.623014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:50.873943 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:50.873827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:51.170998 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:51.170894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:51.297098 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:51.297067 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:51.397926 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:51.397896 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:51.500707 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:51.500670 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:57.696194 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:57.696158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:57.800245 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:57.800211 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:57.902240 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:57.902210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:30:58.081212 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:30:58.081178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:04.382926 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:04.382885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:04.454093 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:04.454057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:11.349315 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:11.349275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:11.707669 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:11.707587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:12.174414 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:12.174379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:14.294581 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:14.294546 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:14.390382 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:14.390353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:14.490932 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:14.490897 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:15.075241 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:15.075207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:15.174616 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:15.174581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:17.103105 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:17.103065 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:31:18.747588 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:18.747552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:28.753499 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:28.753466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:31:31.487648 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:31.487613 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:31:42.988727 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:31:42.988646 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:32:01.198928 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:01.198888 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:32:02.617018 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:02.616982 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:07.147639 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:07.147604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:09.392706 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:09.392656 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:12.090842 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:12.090801 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:32:14.470187 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:14.470155 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:18.727468 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:18.727432 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:18.832036 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:18.831999 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:20.913419 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:20.913382 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:50.874830 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:50.874797 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:51.714903 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:51.714873 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:51.820296 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:51.820255 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:32:52.435617 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:32:52.435581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:07.880439 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:07.880350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:09.261358 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:09.261322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:09.534455 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:09.534366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:09.631174 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:09.631142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:09.734714 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:09.734679 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:10.473462 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:10.473427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:10.560094 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:10.560062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:12.056853 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:12.056817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:12.297167 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:12.297128 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:33:18.863005 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:18.862974 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:18.946896 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:18.946862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:19.536056 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:19.536019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:22.287377 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:22.287338 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:33:26.005502 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:26.005468 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:26.114744 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:26.114714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:26.216588 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:26.216552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:36.754876 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:36.754841 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:37.118360 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:37.118271 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:33:58.156088 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:33:58.156057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:05.016428 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:05.016392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:05.115882 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:05.115849 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:05.373130 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:05.373037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:05.797955 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:05.797897 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:05.978484 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:05.978449 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:09.303075 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:09.303038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:09.403364 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:09.403332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:09.532984 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:09.532950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:10.006780 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:10.006714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:10.240168 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:10.240133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:10.338476 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:10.338399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:13.785473 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:13.785431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:14.125051 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:14.124963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:14.430385 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:14.430302 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:14.645354 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:14.645319 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:15.098667 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:15.098635 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:34:17.453584 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:17.453548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:17.550410 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:17.550364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:17.637289 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:17.637260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:18.026414 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:18.026368 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:18.187120 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:18.187085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:18.277113 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:18.277018 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:21.909160 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:21.909119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:31.071698 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.071667 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:31.172873 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.172839 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:31.409588 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.409499 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:34:31.795073 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.795036 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:34:31.798416 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.798397 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.801816 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.801794 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-be4c79\"/\"kube-root-ca.crt\"" Jul 17 14:34:31.801938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.801833 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-be4c79\"/\"default-dockercfg-z9pdc\"" Jul 17 14:34:31.801938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.801795 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-be4c79\"/\"test-model-be4c79-kserve-self-signed-certs\"" Jul 17 14:34:31.801938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.801795 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-be4c79\"/\"openshift-service-ca.crt\"" Jul 17 14:34:31.807938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.807915 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:34:31.825031 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.825001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:31.919968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.919929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.919968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.919967 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.920197 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.919995 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.920197 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.920094 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.920197 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.920121 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsxzh\" (UniqueName: \"kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:31.920197 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:31.920195 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021317 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021276 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021317 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021569 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021335 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021569 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021361 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021569 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021569 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsxzh\" (UniqueName: \"kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021811 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021784 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021875 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.021970 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.021947 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.023646 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.023625 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.023824 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.023808 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.031095 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.031061 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsxzh\" (UniqueName: \"kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh\") pod \"test-model-be4c79-kserve-5fb98c69d5-6mx5d\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.108280 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.108179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:32.243829 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.243801 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:34:32.245492 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:34:32.245457 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod227c6b89_3cb8_4088_8104_7b30ebf19d49.slice/crio-319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29 WatchSource:0}: Error finding container 319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29: Status 404 returned error can't find the container with id 319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29 Jul 17 14:34:32.700185 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.700150 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" event={"ID":"227c6b89-3cb8-4088-8104-7b30ebf19d49","Type":"ContainerStarted","Data":"20a0b55c44ad12c9b570e688388e6cb9e962b82223494383737050c68d404bc6"} Jul 17 14:34:32.700400 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.700195 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" event={"ID":"227c6b89-3cb8-4088-8104-7b30ebf19d49","Type":"ContainerStarted","Data":"319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29"} Jul 17 14:34:32.722717 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:32.722668 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" podStartSLOduration=1.7226508740000002 podStartE2EDuration="1.722650874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:34:32.722180803 +0000 UTC m=+2959.483045817" watchObservedRunningTime="2026-07-17 14:34:32.722650874 +0000 UTC m=+2959.483515886" Jul 17 14:34:39.005585 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:39.005497 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:39.105509 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:39.105478 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:34:42.109150 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:42.109111 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:42.109613 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:42.109164 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:42.121468 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:42.121448 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:34:42.182819 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:42.182790 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:34:42.756526 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:34:42.756498 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:35:07.933576 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:07.933538 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:13.692299 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:13.692259 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:35:30.771502 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:30.771463 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:35:30.787555 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:30.787524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:31.888634 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:31.888599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:32.578936 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:32.578903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:32.679815 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:32.679782 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:39.102639 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:39.102598 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:35:42.558287 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:42.558248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:45.791383 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:45.791323 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" podUID="227c6b89-3cb8-4088-8104-7b30ebf19d49" containerName="main" containerID="cri-o://20a0b55c44ad12c9b570e688388e6cb9e962b82223494383737050c68d404bc6" gracePeriod=45 Jul 17 14:35:46.038843 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.038809 2555 generic.go:356] "Generic (PLEG): container finished" podID="227c6b89-3cb8-4088-8104-7b30ebf19d49" containerID="20a0b55c44ad12c9b570e688388e6cb9e962b82223494383737050c68d404bc6" exitCode=0 Jul 17 14:35:46.038997 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.038874 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" event={"ID":"227c6b89-3cb8-4088-8104-7b30ebf19d49","Type":"ContainerDied","Data":"20a0b55c44ad12c9b570e688388e6cb9e962b82223494383737050c68d404bc6"} Jul 17 14:35:46.038997 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.038919 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" event={"ID":"227c6b89-3cb8-4088-8104-7b30ebf19d49","Type":"ContainerDied","Data":"319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29"} Jul 17 14:35:46.038997 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.038935 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="319642ec3c1b1abfa7f02448b3ed0c305044083fc8860dfb948bd44aca50be29" Jul 17 14:35:46.042002 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.041947 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:35:46.126982 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.126952 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127139 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.126999 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127139 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127033 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsxzh\" (UniqueName: \"kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127139 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127052 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127139 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127097 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127139 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127115 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir\") pod \"227c6b89-3cb8-4088-8104-7b30ebf19d49\" (UID: \"227c6b89-3cb8-4088-8104-7b30ebf19d49\") " Jul 17 14:35:46.127417 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127267 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home" (OuterVolumeSpecName: "home") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:35:46.127417 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127279 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache" (OuterVolumeSpecName: "model-cache") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:35:46.127512 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.127427 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:35:46.129415 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.129383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm" (OuterVolumeSpecName: "dshm") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:35:46.129520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.129415 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:35:46.129520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.129497 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh" (OuterVolumeSpecName: "kube-api-access-dsxzh") pod "227c6b89-3cb8-4088-8104-7b30ebf19d49" (UID: "227c6b89-3cb8-4088-8104-7b30ebf19d49"). InnerVolumeSpecName "kube-api-access-dsxzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:35:46.228291 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228255 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:46.228291 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228287 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:46.228520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228301 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsxzh\" (UniqueName: \"kubernetes.io/projected/227c6b89-3cb8-4088-8104-7b30ebf19d49-kube-api-access-dsxzh\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:46.228520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228310 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/227c6b89-3cb8-4088-8104-7b30ebf19d49-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:46.228520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228318 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:46.228520 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:46.228328 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227c6b89-3cb8-4088-8104-7b30ebf19d49-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:35:47.044576 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.044544 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d" Jul 17 14:35:47.078487 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.078432 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:35:47.080181 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.080157 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-be4c79/test-model-be4c79-kserve-5fb98c69d5-6mx5d"] Jul 17 14:35:47.116651 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.116615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:47.551965 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.551929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:47.678569 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:47.678530 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="227c6b89-3cb8-4088-8104-7b30ebf19d49" path="/var/lib/kubelet/pods/227c6b89-3cb8-4088-8104-7b30ebf19d49/volumes" Jul 17 14:35:51.794653 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:51.794618 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:35:53.939209 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:53.939173 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:54.036451 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:54.036420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:54.327155 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:54.327115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:54.454408 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:54.454354 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:55.127015 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:55.126969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:35:55.222026 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:35:55.221996 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:05.089191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:05.089153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:07.886600 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:07.886562 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:36:09.755250 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:09.755212 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:10.126224 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:10.126137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:11.454801 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:11.454742 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:11.555450 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:11.555413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:16.410065 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:16.410030 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:16.488634 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:16.488603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:16.675805 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:16.675711 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:16.903366 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:16.903335 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:17.323809 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:17.323735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:17.638113 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:17.638025 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:17.903852 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:17.903769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:21.098577 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:21.098544 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:21.200187 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:21.200159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:21.298821 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:21.298785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:21.457475 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:21.457392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:21.904809 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:21.904775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:22.005196 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:22.005160 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:25.567407 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:25.567376 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:26.036528 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:26.036490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:26.435205 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:26.435116 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:26.630805 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:26.630770 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:26.799533 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:26.799496 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:36:29.215054 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:29.215017 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:29.348443 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:29.348412 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:29.439401 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:29.439363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:29.940526 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:29.940495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:30.173768 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:30.173730 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:30.269291 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:30.269257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:33.642885 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:33.642852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:42.263970 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:42.263942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:42.365380 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:42.365341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:45.125492 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:45.125454 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:36:51.026295 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:51.026259 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:51.125278 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:51.125242 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:36:56.869354 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:36:56.869322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:00.197560 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:00.197519 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:37:01.790402 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:01.790367 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:02.162260 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:02.162173 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:08.504967 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:08.504934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:08.603464 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:08.603437 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:08.817487 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:08.817402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:09.022064 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:09.022036 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:24.777498 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:24.777460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:29.245366 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:29.245332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:29.593848 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:29.593744 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:37:29.638878 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:29.638853 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:36.806191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:36.806110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:36.954068 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:36.954036 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:37.304199 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:37.304170 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:37.510105 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:37.510073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:37.755610 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:37.755578 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:38.153071 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:38.152984 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:38.652817 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:38.652776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:41.678423 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:41.678388 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:41.783466 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:41.783435 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:41.947022 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:41.946930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:42.282221 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:42.282194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:42.935930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:42.935895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:43.034808 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:43.034776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:46.193278 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:46.193244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:46.539714 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:46.539681 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:46.957004 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:46.956922 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:47.079597 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:47.079568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:47.495239 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:47.495203 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:37:49.775550 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:49.775516 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:49.901040 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:49.900999 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:50.003041 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:50.003009 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:50.148425 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:50.148339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:50.885462 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:50.885350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:51.010154 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:51.010118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:52.642653 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:52.642622 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:52.748026 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:52.747996 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:37:54.202143 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:37:54.202112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:03.490887 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:03.490852 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:38:04.052391 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:04.052347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:04.151801 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:04.151738 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:06.756411 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.756379 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:38:06.756827 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.756657 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="227c6b89-3cb8-4088-8104-7b30ebf19d49" containerName="main" Jul 17 14:38:06.756827 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.756670 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="227c6b89-3cb8-4088-8104-7b30ebf19d49" containerName="main" Jul 17 14:38:06.756827 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.756738 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="227c6b89-3cb8-4088-8104-7b30ebf19d49" containerName="main" Jul 17 14:38:06.759655 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.759637 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.763708 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.763688 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"kube-root-ca.crt\"" Jul 17 14:38:06.763863 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.763844 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"openshift-service-ca.crt\"" Jul 17 14:38:06.763922 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.763847 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"auth-test-model-244a73-kserve-self-signed-certs\"" Jul 17 14:38:06.763922 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.763895 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"default-dockercfg-hl8w5\"" Jul 17 14:38:06.767980 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.767957 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:06.772971 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.772947 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:38:06.839913 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.839877 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.840092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.839929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.840092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.839992 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.840092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.840038 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.840092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.840061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.840233 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.840095 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kknbg\" (UniqueName: \"kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941275 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941238 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941275 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941463 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941295 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941463 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941463 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kknbg\" (UniqueName: \"kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941463 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941405 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941743 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941716 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941845 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941745 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.941904 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.941883 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.943621 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.943599 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.944003 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.943985 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:06.951075 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:06.951054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kknbg\" (UniqueName: \"kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg\") pod \"auth-test-model-244a73-kserve-bdfb788f9-rr672\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:07.068959 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:07.068875 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:07.209342 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:07.209312 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:38:07.212064 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:38:07.212033 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73a2c1da_6e9b_4176_a46c_b3776adb5b2b.slice/crio-646630a5c89ff5f108311501072c0e8766787607b0ffcee0a37d5c283c48263d WatchSource:0}: Error finding container 646630a5c89ff5f108311501072c0e8766787607b0ffcee0a37d5c283c48263d: Status 404 returned error can't find the container with id 646630a5c89ff5f108311501072c0e8766787607b0ffcee0a37d5c283c48263d Jul 17 14:38:07.687668 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:07.687637 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" event={"ID":"73a2c1da-6e9b-4176-a46c-b3776adb5b2b","Type":"ContainerStarted","Data":"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06"} Jul 17 14:38:07.687668 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:07.687675 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" event={"ID":"73a2c1da-6e9b-4176-a46c-b3776adb5b2b","Type":"ContainerStarted","Data":"646630a5c89ff5f108311501072c0e8766787607b0ffcee0a37d5c283c48263d"} Jul 17 14:38:07.732456 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:07.732398 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" podStartSLOduration=1.7323800120000001 podStartE2EDuration="1.732380012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:38:07.731426248 +0000 UTC m=+3174.492291263" watchObservedRunningTime="2026-07-17 14:38:07.732380012 +0000 UTC m=+3174.493245022" Jul 17 14:38:11.780925 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:11.780894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:12.787919 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:12.787886 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:17.069155 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:17.069109 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:17.069649 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:17.069170 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:17.081718 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:17.081685 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:17.743772 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:17.743726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:38:18.595267 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:18.595234 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:38:21.225818 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:21.225789 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:28.500014 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:28.499979 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:35.299213 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.299178 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:38:35.302366 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.302348 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.304890 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.304869 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"kube-root-ca.crt\"" Jul 17 14:38:35.305704 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.305685 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"openshift-service-ca.crt\"" Jul 17 14:38:35.305834 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.305721 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"auth-test-model-9eb655-kserve-self-signed-certs\"" Jul 17 14:38:35.305834 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.305722 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"default-dockercfg-mktv9\"" Jul 17 14:38:35.316952 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.316925 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:38:35.339157 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.339120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:38:35.391512 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391472 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.391676 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.391676 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391551 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.391676 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391586 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.391831 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391668 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6c6\" (UniqueName: \"kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.391831 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.391712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.492987 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.492945 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.492987 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.492991 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493200 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493200 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493034 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493200 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6c6\" (UniqueName: \"kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493324 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493232 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493444 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493425 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493525 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493489 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.493610 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.493591 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.495436 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.495411 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.495589 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.495571 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.503397 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.503369 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6c6\" (UniqueName: \"kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6\") pod \"auth-test-model-9eb655-kserve-5c7c447779-2qvgp\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.611011 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.610906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:35.746303 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.746275 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:38:35.748520 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:38:35.748492 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadaf4c69_826e_4c12_b5db_9f068b0ddad6.slice/crio-09c3931a3c3896cc55d972849703cc834353f94a6c8bd6aabb258c3307a3cdaf WatchSource:0}: Error finding container 09c3931a3c3896cc55d972849703cc834353f94a6c8bd6aabb258c3307a3cdaf: Status 404 returned error can't find the container with id 09c3931a3c3896cc55d972849703cc834353f94a6c8bd6aabb258c3307a3cdaf Jul 17 14:38:35.815880 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.815845 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" event={"ID":"adaf4c69-826e-4c12-b5db-9f068b0ddad6","Type":"ContainerStarted","Data":"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b"} Jul 17 14:38:35.815880 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.815886 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" event={"ID":"adaf4c69-826e-4c12-b5db-9f068b0ddad6","Type":"ContainerStarted","Data":"09c3931a3c3896cc55d972849703cc834353f94a6c8bd6aabb258c3307a3cdaf"} Jul 17 14:38:35.836208 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:35.836141 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" podStartSLOduration=0.836122046 podStartE2EDuration="836.122046ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:38:35.833245168 +0000 UTC m=+3202.594110181" watchObservedRunningTime="2026-07-17 14:38:35.836122046 +0000 UTC m=+3202.596987058" Jul 17 14:38:39.593988 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:39.593953 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:38:45.611180 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:45.611139 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:45.611579 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:45.611194 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:45.623597 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:45.623568 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:38:45.872423 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:38:45.872331 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:39:08.613269 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:08.613184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:39:10.125938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:10.125898 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:39:27.578247 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:27.578214 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:39:46.086102 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:46.086056 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:39:46.524423 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:46.524387 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:39:46.595962 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:39:46.595925 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:40:00.486211 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:00.486174 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:40:18.087253 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:18.087218 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:40:37.582831 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:37.582734 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:40:52.630056 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:52.630017 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:40:52.655150 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:52.655121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:40:53.686240 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:53.686208 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:40:53.705807 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:53.705777 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:40:56.301954 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:40:56.301914 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:41:06.305620 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:06.305585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:07.651950 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.651911 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerName="main" containerID="cri-o://61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06" gracePeriod=45 Jul 17 14:41:07.733065 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.733029 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.33:8000/ready\": dial tcp 10.132.0.33:8000: connect: connection refused" Jul 17 14:41:07.895023 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.894996 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:41:07.987412 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987315 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987412 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987353 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987412 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987369 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987412 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987387 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987781 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987423 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kknbg\" (UniqueName: \"kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987781 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987450 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache\") pod \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\" (UID: \"73a2c1da-6e9b-4176-a46c-b3776adb5b2b\") " Jul 17 14:41:07.987781 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987684 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home" (OuterVolumeSpecName: "home") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:07.987781 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987701 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:07.987937 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.987818 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache" (OuterVolumeSpecName: "model-cache") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:07.989684 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.989656 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm" (OuterVolumeSpecName: "dshm") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:07.989684 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.989662 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg" (OuterVolumeSpecName: "kube-api-access-kknbg") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "kube-api-access-kknbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:41:07.989684 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:07.989653 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73a2c1da-6e9b-4176-a46c-b3776adb5b2b" (UID: "73a2c1da-6e9b-4176-a46c-b3776adb5b2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:41:08.088618 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088577 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.088618 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088609 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.088618 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088621 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.088904 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088633 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.088904 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088646 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kknbg\" (UniqueName: \"kubernetes.io/projected/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-kube-api-access-kknbg\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.088904 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.088660 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73a2c1da-6e9b-4176-a46c-b3776adb5b2b-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:08.530765 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.530713 2555 generic.go:356] "Generic (PLEG): container finished" podID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerID="61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06" exitCode=0 Jul 17 14:41:08.530938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.530778 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" Jul 17 14:41:08.530938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.530797 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" event={"ID":"73a2c1da-6e9b-4176-a46c-b3776adb5b2b","Type":"ContainerDied","Data":"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06"} Jul 17 14:41:08.530938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.530842 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672" event={"ID":"73a2c1da-6e9b-4176-a46c-b3776adb5b2b","Type":"ContainerDied","Data":"646630a5c89ff5f108311501072c0e8766787607b0ffcee0a37d5c283c48263d"} Jul 17 14:41:08.530938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.530859 2555 scope.go:117] "RemoveContainer" containerID="61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06" Jul 17 14:41:08.540166 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.540147 2555 scope.go:117] "RemoveContainer" containerID="61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06" Jul 17 14:41:08.540460 ip-10-0-133-123 kubenswrapper[2555]: E0717 14:41:08.540440 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06\": container with ID starting with 61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06 not found: ID does not exist" containerID="61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06" Jul 17 14:41:08.540526 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.540473 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06"} err="failed to get container status \"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06\": rpc error: code = NotFound desc = could not find container \"61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06\": container with ID starting with 61cf53d54ade0febaf3403d3bb8a680eeef989e126b9a7c2dd5768a220892a06 not found: ID does not exist" Jul 17 14:41:08.555893 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.555863 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:41:08.560354 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.560323 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/auth-test-model-244a73-kserve-bdfb788f9-rr672"] Jul 17 14:41:08.706434 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.706398 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" podUID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" containerName="main" containerID="cri-o://751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b" gracePeriod=45 Jul 17 14:41:08.953986 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:08.953959 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:41:09.097975 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.097878 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj6c6\" (UniqueName: \"kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.097975 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.097933 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.097975 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.097967 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.098257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.097981 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.098257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.098029 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.098257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.098049 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache\") pod \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\" (UID: \"adaf4c69-826e-4c12-b5db-9f068b0ddad6\") " Jul 17 14:41:09.098257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.098228 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home" (OuterVolumeSpecName: "home") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:09.098462 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.098395 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache" (OuterVolumeSpecName: "model-cache") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:09.098462 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.098407 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:09.100248 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.100223 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm" (OuterVolumeSpecName: "dshm") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:41:09.100358 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.100268 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6" (OuterVolumeSpecName: "kube-api-access-qj6c6") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "kube-api-access-qj6c6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:41:09.100358 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.100264 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "adaf4c69-826e-4c12-b5db-9f068b0ddad6" (UID: "adaf4c69-826e-4c12-b5db-9f068b0ddad6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:41:09.199188 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199150 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.199188 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199184 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.199188 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199193 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.199451 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199201 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.199451 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199210 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adaf4c69-826e-4c12-b5db-9f068b0ddad6-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.199451 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.199219 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj6c6\" (UniqueName: \"kubernetes.io/projected/adaf4c69-826e-4c12-b5db-9f068b0ddad6-kube-api-access-qj6c6\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:41:09.538182 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.538143 2555 generic.go:356] "Generic (PLEG): container finished" podID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" containerID="751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b" exitCode=0 Jul 17 14:41:09.538182 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.538183 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" Jul 17 14:41:09.538415 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.538221 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" event={"ID":"adaf4c69-826e-4c12-b5db-9f068b0ddad6","Type":"ContainerDied","Data":"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b"} Jul 17 14:41:09.538415 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.538255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp" event={"ID":"adaf4c69-826e-4c12-b5db-9f068b0ddad6","Type":"ContainerDied","Data":"09c3931a3c3896cc55d972849703cc834353f94a6c8bd6aabb258c3307a3cdaf"} Jul 17 14:41:09.538415 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.538274 2555 scope.go:117] "RemoveContainer" containerID="751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b" Jul 17 14:41:09.546995 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.546978 2555 scope.go:117] "RemoveContainer" containerID="751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b" Jul 17 14:41:09.547294 ip-10-0-133-123 kubenswrapper[2555]: E0717 14:41:09.547273 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b\": container with ID starting with 751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b not found: ID does not exist" containerID="751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b" Jul 17 14:41:09.547342 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.547303 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b"} err="failed to get container status \"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b\": rpc error: code = NotFound desc = could not find container \"751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b\": container with ID starting with 751574d8d55bd30e4042397016e77284e1d25fb9bc1af993b8f51dea7d50af7b not found: ID does not exist" Jul 17 14:41:09.558870 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.558836 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:41:09.562533 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.562508 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/auth-test-model-9eb655-kserve-5c7c447779-2qvgp"] Jul 17 14:41:09.678661 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.678629 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" path="/var/lib/kubelet/pods/73a2c1da-6e9b-4176-a46c-b3776adb5b2b/volumes" Jul 17 14:41:09.679044 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:09.679030 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" path="/var/lib/kubelet/pods/adaf4c69-826e-4c12-b5db-9f068b0ddad6/volumes" Jul 17 14:41:10.587172 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:10.587135 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:41:13.944435 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:13.944399 2555 scope.go:117] "RemoveContainer" containerID="20a0b55c44ad12c9b570e688388e6cb9e962b82223494383737050c68d404bc6" Jul 17 14:41:15.643359 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:15.643328 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:19.493039 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493009 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493280 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" containerName="main" Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493289 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" containerName="main" Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493306 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerName="main" Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493312 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerName="main" Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493365 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a2c1da-6e9b-4176-a46c-b3776adb5b2b" containerName="main" Jul 17 14:41:19.493421 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.493371 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="adaf4c69-826e-4c12-b5db-9f068b0ddad6" containerName="main" Jul 17 14:41:19.495948 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.495930 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.499773 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.499729 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"default-dockercfg-mktv9\"" Jul 17 14:41:19.499956 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.499884 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"kube-root-ca.crt\"" Jul 17 14:41:19.499956 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.499928 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"sub-test-model-9eb655-kserve-self-signed-certs\"" Jul 17 14:41:19.499956 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.499947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"openshift-service-ca.crt\"" Jul 17 14:41:19.506398 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.506367 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:41:19.608165 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.608132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:19.690866 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.690826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.690866 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.690870 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls8th\" (UniqueName: \"kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.691081 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.690895 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.691081 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.690956 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.691081 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.691036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.691184 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.691091 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.709177 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.709143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:19.791885 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.791885 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792083 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792083 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792083 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791968 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls8th\" (UniqueName: \"kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792083 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.791996 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792390 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.792351 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792470 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.792359 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.792470 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.792442 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.794236 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.794215 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.794398 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.794383 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.801118 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.801091 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls8th\" (UniqueName: \"kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th\") pod \"sub-test-model-9eb655-kserve-74647d8f7f-6mhxc\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.805883 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.805859 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:19.939970 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:19.939940 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:41:19.941850 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:41:19.941816 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb673207_ecc2_4d0b_a148_7baf155705ab.slice/crio-891ab6ecd869483da6b4c690c3f7bb836df3441d0ca464b755a438f670c9c648 WatchSource:0}: Error finding container 891ab6ecd869483da6b4c690c3f7bb836df3441d0ca464b755a438f670c9c648: Status 404 returned error can't find the container with id 891ab6ecd869483da6b4c690c3f7bb836df3441d0ca464b755a438f670c9c648 Jul 17 14:41:20.591227 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:20.591191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" event={"ID":"bb673207-ecc2-4d0b-a148-7baf155705ab","Type":"ContainerStarted","Data":"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1"} Jul 17 14:41:20.591621 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:20.591235 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" event={"ID":"bb673207-ecc2-4d0b-a148-7baf155705ab","Type":"ContainerStarted","Data":"891ab6ecd869483da6b4c690c3f7bb836df3441d0ca464b755a438f670c9c648"} Jul 17 14:41:20.609145 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:20.609091 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" podStartSLOduration=1.6090735619999998 podStartE2EDuration="1.609073562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:41:20.607285277 +0000 UTC m=+3367.368150302" watchObservedRunningTime="2026-07-17 14:41:20.609073562 +0000 UTC m=+3367.369938589" Jul 17 14:41:22.380618 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:22.380585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:23.393790 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:23.393741 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:41:29.806899 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:29.806843 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:29.806899 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:29.806915 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:29.819351 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:29.819323 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:30.649242 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:30.649213 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:41:34.791697 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:34.791663 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:41:44.090559 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:44.090523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:52.690189 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:52.690148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:55.965143 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:55.965103 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:41:55.991366 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:55.991333 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:41:57.688549 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:41:57.688515 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:42:07.222856 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:07.222772 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:42:09.297381 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.297351 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:42:09.300539 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.300522 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.305208 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.305186 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"kube-root-ca.crt\"" Jul 17 14:42:09.306156 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.306128 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"openshift-service-ca.crt\"" Jul 17 14:42:09.306267 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.306161 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"default-dockercfg-hl8w5\"" Jul 17 14:42:09.306267 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.306174 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"rate-test-model-244a73-kserve-self-signed-certs\"" Jul 17 14:42:09.312896 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.312876 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:42:09.344693 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.344654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:42:09.413931 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.413893 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.414100 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.413946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.414100 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.413983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.414100 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.414001 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.414100 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.414035 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxbr\" (UniqueName: \"kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.414257 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.414100 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515261 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515457 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515457 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515457 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515457 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515457 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxbr\" (UniqueName: \"kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515736 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515660 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515736 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515711 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.515866 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.515803 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.517583 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.517562 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.517872 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.517854 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.524215 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.524194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxbr\" (UniqueName: \"kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr\") pod \"rate-test-model-244a73-kserve-bd948c4c9-hzhlj\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.610732 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.610631 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:09.743784 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.743738 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:42:09.746002 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:42:09.745974 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc94394b_c848_4451_8b97_d9b48daa16b3.slice/crio-07623440d5606253527f87e6fb1217b2b18ff6d8855a3e38135a39d342866260 WatchSource:0}: Error finding container 07623440d5606253527f87e6fb1217b2b18ff6d8855a3e38135a39d342866260: Status 404 returned error can't find the container with id 07623440d5606253527f87e6fb1217b2b18ff6d8855a3e38135a39d342866260 Jul 17 14:42:09.824708 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.824675 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" event={"ID":"cc94394b-c848-4451-8b97-d9b48daa16b3","Type":"ContainerStarted","Data":"da7edfd86d9b32690f4a854bb26cc8c3311abaa3adf5d4620f8f136ab64dd3d2"} Jul 17 14:42:09.824910 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.824718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" event={"ID":"cc94394b-c848-4451-8b97-d9b48daa16b3","Type":"ContainerStarted","Data":"07623440d5606253527f87e6fb1217b2b18ff6d8855a3e38135a39d342866260"} Jul 17 14:42:09.841864 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:09.841807 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" podStartSLOduration=0.841784257 podStartE2EDuration="841.784257ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:42:09.84078642 +0000 UTC m=+3416.601651427" watchObservedRunningTime="2026-07-17 14:42:09.841784257 +0000 UTC m=+3416.602649272" Jul 17 14:42:10.984500 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:10.984463 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" podUID="bb673207-ecc2-4d0b-a148-7baf155705ab" containerName="main" containerID="cri-o://2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1" gracePeriod=45 Jul 17 14:42:11.231945 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.231907 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:42:11.233980 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.233942 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234116 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.233993 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234116 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234072 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234116 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234290 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234160 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234290 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234192 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls8th\" (UniqueName: \"kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th\") pod \"bb673207-ecc2-4d0b-a148-7baf155705ab\" (UID: \"bb673207-ecc2-4d0b-a148-7baf155705ab\") " Jul 17 14:42:11.234290 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234216 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home" (OuterVolumeSpecName: "home") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:42:11.234290 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234224 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:42:11.234542 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234382 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.234542 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234398 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.234542 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.234409 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:42:11.236665 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.236635 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm" (OuterVolumeSpecName: "dshm") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:42:11.236665 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.236639 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th" (OuterVolumeSpecName: "kube-api-access-ls8th") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "kube-api-access-ls8th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:42:11.236852 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.236823 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb673207-ecc2-4d0b-a148-7baf155705ab" (UID: "bb673207-ecc2-4d0b-a148-7baf155705ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:42:11.334761 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.334708 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.334944 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.334785 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb673207-ecc2-4d0b-a148-7baf155705ab-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.334944 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.334797 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb673207-ecc2-4d0b-a148-7baf155705ab-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.334944 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.334806 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls8th\" (UniqueName: \"kubernetes.io/projected/bb673207-ecc2-4d0b-a148-7baf155705ab-kube-api-access-ls8th\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:42:11.836055 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.836025 2555 generic.go:356] "Generic (PLEG): container finished" podID="bb673207-ecc2-4d0b-a148-7baf155705ab" containerID="2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1" exitCode=0 Jul 17 14:42:11.836230 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.836095 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" event={"ID":"bb673207-ecc2-4d0b-a148-7baf155705ab","Type":"ContainerDied","Data":"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1"} Jul 17 14:42:11.836230 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.836140 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" event={"ID":"bb673207-ecc2-4d0b-a148-7baf155705ab","Type":"ContainerDied","Data":"891ab6ecd869483da6b4c690c3f7bb836df3441d0ca464b755a438f670c9c648"} Jul 17 14:42:11.836230 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.836153 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc" Jul 17 14:42:11.836354 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.836157 2555 scope.go:117] "RemoveContainer" containerID="2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1" Jul 17 14:42:11.845318 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.845299 2555 scope.go:117] "RemoveContainer" containerID="2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1" Jul 17 14:42:11.845607 ip-10-0-133-123 kubenswrapper[2555]: E0717 14:42:11.845590 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1\": container with ID starting with 2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1 not found: ID does not exist" containerID="2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1" Jul 17 14:42:11.845675 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.845619 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1"} err="failed to get container status \"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1\": rpc error: code = NotFound desc = could not find container \"2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1\": container with ID starting with 2c4bd980c1629cb81189455906e9414eddd98ba2ba78b903f2ee2fe9b49a48f1 not found: ID does not exist" Jul 17 14:42:11.853838 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.853810 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:42:11.859657 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:11.859630 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/sub-test-model-9eb655-kserve-74647d8f7f-6mhxc"] Jul 17 14:42:13.678717 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:13.678685 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb673207-ecc2-4d0b-a148-7baf155705ab" path="/var/lib/kubelet/pods/bb673207-ecc2-4d0b-a148-7baf155705ab/volumes" Jul 17 14:42:19.085488 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:19.085451 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:42:19.611650 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:19.611608 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:19.611879 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:19.611677 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:19.624316 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:19.624273 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:19.890401 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:19.890305 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:42:53.188741 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.188699 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:42:53.191128 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.189117 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb673207-ecc2-4d0b-a148-7baf155705ab" containerName="main" Jul 17 14:42:53.191128 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.189137 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb673207-ecc2-4d0b-a148-7baf155705ab" containerName="main" Jul 17 14:42:53.191128 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.189219 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb673207-ecc2-4d0b-a148-7baf155705ab" containerName="main" Jul 17 14:42:53.192028 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.192003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.194702 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.194677 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"openshift-service-ca.crt\"" Jul 17 14:42:53.195837 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.195815 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"default-dockercfg-mktv9\"" Jul 17 14:42:53.195933 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.195823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"rate-test-model-9eb655-kserve-self-signed-certs\"" Jul 17 14:42:53.195933 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.195863 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9eb655\"/\"kube-root-ca.crt\"" Jul 17 14:42:53.207993 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.207962 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:42:53.239996 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.239963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:42:53.262131 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262088 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.262131 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262132 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vpnb\" (UniqueName: \"kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.262342 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262159 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.262342 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262256 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.262342 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.262463 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.262350 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363290 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363328 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363379 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vpnb\" (UniqueName: \"kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363784 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363720 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363851 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363801 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.363894 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.363847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.365665 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.365647 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.365950 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.365934 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.371804 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.371780 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vpnb\" (UniqueName: \"kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb\") pod \"rate-test-model-9eb655-kserve-849fd4d567-mkgbf\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.501204 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.501167 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:42:53.562065 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.562029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:42:53.638259 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.638230 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:42:53.639806 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:42:53.639776 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51dd8cbd_59ca_4952_9b95_a9ecd68ace04.slice/crio-ee8abec6f13a75c38e0b6b95955afed5cf330b1bf4888d33f424fb22c250e9bd WatchSource:0}: Error finding container ee8abec6f13a75c38e0b6b95955afed5cf330b1bf4888d33f424fb22c250e9bd: Status 404 returned error can't find the container with id ee8abec6f13a75c38e0b6b95955afed5cf330b1bf4888d33f424fb22c250e9bd Jul 17 14:42:53.640974 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:53.640947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:42:54.036817 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:54.036784 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" event={"ID":"51dd8cbd-59ca-4952-9b95-a9ecd68ace04","Type":"ContainerStarted","Data":"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7"} Jul 17 14:42:54.036817 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:54.036824 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" event={"ID":"51dd8cbd-59ca-4952-9b95-a9ecd68ace04","Type":"ContainerStarted","Data":"ee8abec6f13a75c38e0b6b95955afed5cf330b1bf4888d33f424fb22c250e9bd"} Jul 17 14:42:54.056548 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:54.056492 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" podStartSLOduration=1.056474702 podStartE2EDuration="1.056474702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:42:54.054337424 +0000 UTC m=+3460.815202437" watchObservedRunningTime="2026-07-17 14:42:54.056474702 +0000 UTC m=+3460.817339733" Jul 17 14:42:56.985385 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:42:56.985351 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:43:03.501872 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:03.501836 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:43:03.501872 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:03.501879 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:43:03.514159 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:03.514131 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:43:04.097778 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:04.097725 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:43:04.814685 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:04.814652 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:04.913316 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:04.913285 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:12.716938 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:12.716901 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:14.355662 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:14.355631 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:21.702014 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:21.701975 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:43:37.004053 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:37.003967 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:43:45.183863 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:45.183830 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:43:45.193233 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:45.193203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:46.818032 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:46.817997 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:43:46.827742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:46.827716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:43:58.103009 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:43:58.102973 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:44:00.203873 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.203806 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" podUID="cc94394b-c848-4451-8b97-d9b48daa16b3" containerName="main" containerID="cri-o://da7edfd86d9b32690f4a854bb26cc8c3311abaa3adf5d4620f8f136ab64dd3d2" gracePeriod=45 Jul 17 14:44:00.351099 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.351066 2555 generic.go:356] "Generic (PLEG): container finished" podID="cc94394b-c848-4451-8b97-d9b48daa16b3" containerID="da7edfd86d9b32690f4a854bb26cc8c3311abaa3adf5d4620f8f136ab64dd3d2" exitCode=0 Jul 17 14:44:00.351266 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.351147 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" event={"ID":"cc94394b-c848-4451-8b97-d9b48daa16b3","Type":"ContainerDied","Data":"da7edfd86d9b32690f4a854bb26cc8c3311abaa3adf5d4620f8f136ab64dd3d2"} Jul 17 14:44:00.446295 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.446267 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:44:00.538005 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.537963 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538041 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538077 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gxbr\" (UniqueName: \"kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538100 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538191 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538162 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538398 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538219 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm\") pod \"cc94394b-c848-4451-8b97-d9b48daa16b3\" (UID: \"cc94394b-c848-4451-8b97-d9b48daa16b3\") " Jul 17 14:44:00.538398 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home" (OuterVolumeSpecName: "home") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:00.538511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538412 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:00.538511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538470 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:00.538511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.538481 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:00.540237 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.540217 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm" (OuterVolumeSpecName: "dshm") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:00.540374 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.540354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:44:00.540469 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.540445 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr" (OuterVolumeSpecName: "kube-api-access-6gxbr") pod "cc94394b-c848-4451-8b97-d9b48daa16b3" (UID: "cc94394b-c848-4451-8b97-d9b48daa16b3"). InnerVolumeSpecName "kube-api-access-6gxbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:44:00.639964 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.639920 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:00.639964 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.639954 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:00.639964 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.639965 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc94394b-c848-4451-8b97-d9b48daa16b3-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:00.639964 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.639974 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gxbr\" (UniqueName: \"kubernetes.io/projected/cc94394b-c848-4451-8b97-d9b48daa16b3-kube-api-access-6gxbr\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:00.640296 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:00.639984 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc94394b-c848-4451-8b97-d9b48daa16b3-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:01.357329 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.357302 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" Jul 17 14:44:01.357730 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.357298 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj" event={"ID":"cc94394b-c848-4451-8b97-d9b48daa16b3","Type":"ContainerDied","Data":"07623440d5606253527f87e6fb1217b2b18ff6d8855a3e38135a39d342866260"} Jul 17 14:44:01.357730 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.357426 2555 scope.go:117] "RemoveContainer" containerID="da7edfd86d9b32690f4a854bb26cc8c3311abaa3adf5d4620f8f136ab64dd3d2" Jul 17 14:44:01.382687 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.382652 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:44:01.388630 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.388606 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/rate-test-model-244a73-kserve-bd948c4c9-hzhlj"] Jul 17 14:44:01.679229 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.679148 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc94394b-c848-4451-8b97-d9b48daa16b3" path="/var/lib/kubelet/pods/cc94394b-c848-4451-8b97-d9b48daa16b3/volumes" Jul 17 14:44:01.837901 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:01.837831 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" podUID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" containerName="main" containerID="cri-o://be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7" gracePeriod=45 Jul 17 14:44:02.084899 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.084876 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:44:02.153659 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153620 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.153659 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153666 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vpnb\" (UniqueName: \"kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.153968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153707 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.153968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153789 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.153968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153819 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.153968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153845 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir\") pod \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\" (UID: \"51dd8cbd-59ca-4952-9b95-a9ecd68ace04\") " Jul 17 14:44:02.154172 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.153992 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache" (OuterVolumeSpecName: "model-cache") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:02.154172 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.154125 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-model-cache\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.154172 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.154146 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home" (OuterVolumeSpecName: "home") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:02.154330 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.154235 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:02.156005 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.155980 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 14:44:02.156112 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.156001 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb" (OuterVolumeSpecName: "kube-api-access-2vpnb") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "kube-api-access-2vpnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:44:02.156112 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.155980 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm" (OuterVolumeSpecName: "dshm") pod "51dd8cbd-59ca-4952-9b95-a9ecd68ace04" (UID: "51dd8cbd-59ca-4952-9b95-a9ecd68ace04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 14:44:02.254538 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.254504 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vpnb\" (UniqueName: \"kubernetes.io/projected/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-kube-api-access-2vpnb\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.254538 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.254537 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-dshm\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.254734 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.254551 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tls-certs\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.254734 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.254560 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-home\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.254734 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.254567 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51dd8cbd-59ca-4952-9b95-a9ecd68ace04-tmp-dir\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:02.363075 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.363037 2555 generic.go:356] "Generic (PLEG): container finished" podID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" containerID="be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7" exitCode=0 Jul 17 14:44:02.363511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.363084 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" Jul 17 14:44:02.363511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.363130 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" event={"ID":"51dd8cbd-59ca-4952-9b95-a9ecd68ace04","Type":"ContainerDied","Data":"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7"} Jul 17 14:44:02.363511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.363176 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf" event={"ID":"51dd8cbd-59ca-4952-9b95-a9ecd68ace04","Type":"ContainerDied","Data":"ee8abec6f13a75c38e0b6b95955afed5cf330b1bf4888d33f424fb22c250e9bd"} Jul 17 14:44:02.363511 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.363202 2555 scope.go:117] "RemoveContainer" containerID="be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7" Jul 17 14:44:02.371922 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.371905 2555 scope.go:117] "RemoveContainer" containerID="be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7" Jul 17 14:44:02.372192 ip-10-0-133-123 kubenswrapper[2555]: E0717 14:44:02.372175 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7\": container with ID starting with be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7 not found: ID does not exist" containerID="be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7" Jul 17 14:44:02.372258 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.372206 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7"} err="failed to get container status \"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7\": rpc error: code = NotFound desc = could not find container \"be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7\": container with ID starting with be1dbebda7112cadc49c59563b0102a2bf99e57d16029eac5032709580b55ab7 not found: ID does not exist" Jul 17 14:44:02.385171 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.385145 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:44:02.388907 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:02.388887 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-9eb655/rate-test-model-9eb655-kserve-849fd4d567-mkgbf"] Jul 17 14:44:03.101573 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101536 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031008929088"] Jul 17 14:44:03.101849 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101836 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" containerName="main" Jul 17 14:44:03.101849 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101850 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" containerName="main" Jul 17 14:44:03.101941 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101867 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc94394b-c848-4451-8b97-d9b48daa16b3" containerName="main" Jul 17 14:44:03.101941 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101873 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc94394b-c848-4451-8b97-d9b48daa16b3" containerName="main" Jul 17 14:44:03.101941 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101919 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" containerName="main" Jul 17 14:44:03.101941 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.101928 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc94394b-c848-4451-8b97-d9b48daa16b3" containerName="main" Jul 17 14:44:03.106061 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.106044 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:03.108847 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.108825 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4cxcv\"" Jul 17 14:44:03.114215 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.114186 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"] Jul 17 14:44:03.263738 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.263701 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9s8w\" (UniqueName: \"kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w\") pod \"test-curl-7683-140031008929088\" (UID: \"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3\") " pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:03.364300 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.364201 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9s8w\" (UniqueName: \"kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w\") pod \"test-curl-7683-140031008929088\" (UID: \"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3\") " pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:03.374186 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.374160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9s8w\" (UniqueName: \"kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w\") pod \"test-curl-7683-140031008929088\" (UID: \"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3\") " pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:03.416224 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.416173 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:03.550279 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.550254 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"] Jul 17 14:44:03.552292 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:44:03.552260 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc10058f_8ac6_4d50_ae56_9f54f7c9a3f3.slice/crio-8afb19acde9c0db026f02d7f61b8a89372b03ab317a1f2a9069a4a838e85e310 WatchSource:0}: Error finding container 8afb19acde9c0db026f02d7f61b8a89372b03ab317a1f2a9069a4a838e85e310: Status 404 returned error can't find the container with id 8afb19acde9c0db026f02d7f61b8a89372b03ab317a1f2a9069a4a838e85e310 Jul 17 14:44:03.553526 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.553509 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 14:44:03.678073 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:03.677991 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51dd8cbd-59ca-4952-9b95-a9ecd68ace04" path="/var/lib/kubelet/pods/51dd8cbd-59ca-4952-9b95-a9ecd68ace04/volumes" Jul 17 14:44:04.376488 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:04.376454 2555 generic.go:356] "Generic (PLEG): container finished" podID="dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" containerID="1a523d343e583bc5dc1f10fda41a357425f2831ab9ed17e6903174c64732f424" exitCode=0 Jul 17 14:44:04.376884 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:04.376502 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008929088" event={"ID":"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3","Type":"ContainerDied","Data":"1a523d343e583bc5dc1f10fda41a357425f2831ab9ed17e6903174c64732f424"} Jul 17 14:44:04.376884 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:04.376535 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008929088" event={"ID":"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3","Type":"ContainerStarted","Data":"8afb19acde9c0db026f02d7f61b8a89372b03ab317a1f2a9069a4a838e85e310"} Jul 17 14:44:05.510407 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.510381 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:05.532387 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.532359 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031008929088_dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3/test-curl-7683-140031008929088/0.log" Jul 17 14:44:05.566266 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.566229 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"] Jul 17 14:44:05.570889 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.570865 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"] Jul 17 14:44:05.585218 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.585189 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9s8w\" (UniqueName: \"kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w\") pod \"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3\" (UID: \"dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3\") " Jul 17 14:44:05.587464 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.587427 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w" (OuterVolumeSpecName: "kube-api-access-j9s8w") pod "dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" (UID: "dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3"). InnerVolumeSpecName "kube-api-access-j9s8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 14:44:05.678186 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.678094 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" path="/var/lib/kubelet/pods/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3/volumes" Jul 17 14:44:05.685991 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:05.685968 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9s8w\" (UniqueName: \"kubernetes.io/projected/dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3-kube-api-access-j9s8w\") on node \"ip-10-0-133-123.ec2.internal\" DevicePath \"\"" Jul 17 14:44:06.386721 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:06.386694 2555 scope.go:117] "RemoveContainer" containerID="1a523d343e583bc5dc1f10fda41a357425f2831ab9ed17e6903174c64732f424" Jul 17 14:44:06.386721 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:06.386708 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088" Jul 17 14:44:06.491944 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:06.491911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:06.590947 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:06.590914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:13.231441 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:13.231390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:13.296585 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:13.296547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:16.314350 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:16.314315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:19.581222 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:19.581187 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:44:20.850458 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:20.850427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:21.291098 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:21.291054 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:22.373444 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:22.373412 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:27.832321 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:27.832282 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:27.933075 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:27.933031 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:28.136849 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:28.136728 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:28.440861 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:28.440777 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:29.137376 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:29.137328 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:29.242939 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:29.242892 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:39.097092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:39.097055 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:42.094493 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:42.094453 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f4hw7"] Jul 17 14:44:43.631378 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:43.631345 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:44.034350 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:44.034315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:50.427152 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:50.427118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:50.509126 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:50.509089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:50.777905 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:50.777863 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:50.983205 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:50.983161 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:44:57.780298 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:57.780268 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-f5d4v_37282188-cdde-4775-9f2e-c25c9b1a95e0/copy-manifests/0.log" Jul 17 14:44:57.788663 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:57.788635 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-f5d4v_37282188-cdde-4775-9f2e-c25c9b1a95e0/manager/0.log" Jul 17 14:44:59.780921 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.780889 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/util/0.log" Jul 17 14:44:59.787870 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.787848 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/pull/0.log" Jul 17 14:44:59.794720 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.794702 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/extract/0.log" Jul 17 14:44:59.917352 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.917326 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/util/0.log" Jul 17 14:44:59.923810 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.923782 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/pull/0.log" Jul 17 14:44:59.929948 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:44:59.929929 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/extract/0.log" Jul 17 14:45:00.044003 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.043935 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/util/0.log" Jul 17 14:45:00.051056 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.051032 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/pull/0.log" Jul 17 14:45:00.057843 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.057821 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/extract/0.log" Jul 17 14:45:00.169965 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.169934 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/util/0.log" Jul 17 14:45:00.177127 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.177094 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/pull/0.log" Jul 17 14:45:00.184472 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.184450 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/extract/0.log" Jul 17 14:45:00.654244 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:00.654215 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jtzj5_a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a/kuadrant-console-plugin/0.log" Jul 17 14:45:01.043011 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:01.042983 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f4hw7_7e571386-1ac9-4dbd-b949-76f56f66713d/limitador/0.log" Jul 17 14:45:01.776477 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:01.776446 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-tp54t_ac4f2a5b-43bb-4e14-a62c-f8a19e93062e/kube-auth-proxy/0.log" Jul 17 14:45:03.105078 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:03.105045 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgv86g_420b1987-3a97-4d23-9405-6d32044e2675/main/0.log" Jul 17 14:45:03.227881 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:03.227847 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-swhrj_ff0487e9-50bd-4e2e-8485-8414cc3e9a2b/main/0.log" Jul 17 14:45:03.353149 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:03.353116 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-bbrtp_eaff2e3d-a06d-48ba-8750-f0f099aa8098/main/0.log" Jul 17 14:45:09.767251 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:09.767153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:45:09.864246 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:09.864199 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" limit=1024 Jul 17 14:45:12.805092 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:12.805060 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r9xs2_58a9967a-28c3-45cd-af44-13d2619266fc/global-pull-secret-syncer/0.log" Jul 17 14:45:12.857022 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:12.856988 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2n5dq_fbcefb70-7a53-4e68-ad3f-6411e521bef8/konnectivity-agent/0.log" Jul 17 14:45:12.978375 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:12.978346 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-123.ec2.internal_f128f886475b532dbbbd78614ba32f76/haproxy/0.log" Jul 17 14:45:17.409788 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.409738 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/extract/0.log" Jul 17 14:45:17.442147 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.442110 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/util/0.log" Jul 17 14:45:17.471356 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.471319 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qnqj8_ef35ea94-30fe-49c5-bf32-fb71626219e0/pull/0.log" Jul 17 14:45:17.507613 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.507577 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/extract/0.log" Jul 17 14:45:17.546143 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.546116 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/util/0.log" Jul 17 14:45:17.566437 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.566411 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zqpqh_4d566442-5ddb-47a0-8405-dd2e3d00d3bb/pull/0.log" Jul 17 14:45:17.600103 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.600075 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/extract/0.log" Jul 17 14:45:17.625454 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.625425 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/util/0.log" Jul 17 14:45:17.653768 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.653728 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739nzqd_b4d43d30-3e5d-4f1d-927c-745d872fd953/pull/0.log" Jul 17 14:45:17.691552 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.691478 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/extract/0.log" Jul 17 14:45:17.727265 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.727227 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/util/0.log" Jul 17 14:45:17.756845 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:17.756815 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ctqm7_bc38f750-3f04-477b-8ff7-dc84a8c4fafa/pull/0.log" Jul 17 14:45:18.342372 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:18.342349 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jtzj5_a20566d9-8bb8-4ccc-9810-aeb5a8e9fe1a/kuadrant-console-plugin/0.log" Jul 17 14:45:18.618968 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:18.618891 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f4hw7_7e571386-1ac9-4dbd-b949-76f56f66713d/limitador/0.log" Jul 17 14:45:20.951126 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:20.951082 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7wkf_a69770f0-9243-45db-bbbe-76567977e50c/node-exporter/0.log" Jul 17 14:45:20.977197 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:20.977153 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7wkf_a69770f0-9243-45db-bbbe-76567977e50c/kube-rbac-proxy/0.log" Jul 17 14:45:21.001080 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:21.001059 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f7wkf_a69770f0-9243-45db-bbbe-76567977e50c/init-textfile/0.log" Jul 17 14:45:22.678034 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.678000 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr"] Jul 17 14:45:22.678448 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.678321 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" containerName="test-curl-7683-140031008929088" Jul 17 14:45:22.678448 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.678332 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" containerName="test-curl-7683-140031008929088" Jul 17 14:45:22.678448 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.678387 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc10058f-8ac6-4d50-ae56-9f54f7c9a3f3" containerName="test-curl-7683-140031008929088" Jul 17 14:45:22.681352 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.681330 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.683915 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.683892 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6g9mf\"/\"default-dockercfg-fhjml\"" Jul 17 14:45:22.684052 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.683928 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6g9mf\"/\"kube-root-ca.crt\"" Jul 17 14:45:22.684724 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.684706 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6g9mf\"/\"openshift-service-ca.crt\"" Jul 17 14:45:22.692869 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.692843 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr"] Jul 17 14:45:22.818567 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.818538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-proc\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.818742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.818576 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-podres\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.818742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.818610 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-lib-modules\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.818742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.818653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-sys\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.818742 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.818703 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbbq\" (UniqueName: \"kubernetes.io/projected/c83f479f-65f0-4e65-8eab-974ef0e7c14d-kube-api-access-wjbbq\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.884361 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.884327 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-j2q6g_ac6513eb-e134-4479-a0c5-2ec3378bc7b1/networking-console-plugin/0.log" Jul 17 14:45:22.919618 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919584 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-proc\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919765 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919625 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-podres\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919765 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919656 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-lib-modules\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919765 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919712 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-proc\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919785 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-lib-modules\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-sys\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919804 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-podres\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919825 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c83f479f-65f0-4e65-8eab-974ef0e7c14d-sys\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.919930 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.919840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbbq\" (UniqueName: \"kubernetes.io/projected/c83f479f-65f0-4e65-8eab-974ef0e7c14d-kube-api-access-wjbbq\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.930433 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.930376 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbbq\" (UniqueName: \"kubernetes.io/projected/c83f479f-65f0-4e65-8eab-974ef0e7c14d-kube-api-access-wjbbq\") pod \"perf-node-gather-daemonset-hsrbr\" (UID: \"c83f479f-65f0-4e65-8eab-974ef0e7c14d\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:22.991376 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:22.991345 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:23.126462 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:23.126431 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr"] Jul 17 14:45:23.128411 ip-10-0-133-123 kubenswrapper[2555]: W0717 14:45:23.128382 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc83f479f_65f0_4e65_8eab_974ef0e7c14d.slice/crio-ec313a8831ec11f5170531915f209b827376fe99cad956f1abc82fc9e9f4bb2a WatchSource:0}: Error finding container ec313a8831ec11f5170531915f209b827376fe99cad956f1abc82fc9e9f4bb2a: Status 404 returned error can't find the container with id ec313a8831ec11f5170531915f209b827376fe99cad956f1abc82fc9e9f4bb2a Jul 17 14:45:23.738099 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:23.738072 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" event={"ID":"c83f479f-65f0-4e65-8eab-974ef0e7c14d","Type":"ContainerStarted","Data":"51c1e606d4a8e8d576e7ad00890f0fe8b115a048a13b45c62095c20758e911eb"} Jul 17 14:45:23.738475 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:23.738119 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" event={"ID":"c83f479f-65f0-4e65-8eab-974ef0e7c14d","Type":"ContainerStarted","Data":"ec313a8831ec11f5170531915f209b827376fe99cad956f1abc82fc9e9f4bb2a"} Jul 17 14:45:23.738475 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:23.738156 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:23.754050 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:23.753997 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" podStartSLOduration=1.753981412 podStartE2EDuration="1.753981412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:45:23.752993485 +0000 UTC m=+3610.513858498" watchObservedRunningTime="2026-07-17 14:45:23.753981412 +0000 UTC m=+3610.514846462" Jul 17 14:45:25.773535 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:25.773508 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vfrhc_7fa85d62-bc6d-437b-a286-ed8ad35ae662/dns/0.log" Jul 17 14:45:25.798127 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:25.798086 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vfrhc_7fa85d62-bc6d-437b-a286-ed8ad35ae662/kube-rbac-proxy/0.log" Jul 17 14:45:25.851035 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:25.851011 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rtgps_181882a0-faee-4bd6-bce1-e4baf72269f8/dns-node-resolver/0.log" Jul 17 14:45:26.497130 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:26.497105 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bfpp_c56b3af1-0414-43b6-a26f-1c8c7ab26463/node-ca/0.log" Jul 17 14:45:27.730555 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:27.730528 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-tp54t_ac4f2a5b-43bb-4e14-a62c-f8a19e93062e/kube-auth-proxy/0.log" Jul 17 14:45:28.691643 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:28.691615 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rq7pc_01d2f7b7-1f92-4d12-99f1-f1524f347729/serve-healthcheck-canary/0.log" Jul 17 14:45:29.400322 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:29.400288 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xk5wr_0c3c34ed-7e58-463b-a19f-6bbbec0a20f9/kube-rbac-proxy/0.log" Jul 17 14:45:29.427313 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:29.427287 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xk5wr_0c3c34ed-7e58-463b-a19f-6bbbec0a20f9/exporter/0.log" Jul 17 14:45:29.455996 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:29.455963 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xk5wr_0c3c34ed-7e58-463b-a19f-6bbbec0a20f9/extractor/0.log" Jul 17 14:45:29.752868 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:29.752841 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-hsrbr" Jul 17 14:45:32.402524 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:32.402444 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-f5d4v_37282188-cdde-4775-9f2e-c25c9b1a95e0/manager/0.log" Jul 17 14:45:32.435959 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:32.435933 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-f5d4v_37282188-cdde-4775-9f2e-c25c9b1a95e0/copy-manifests/0.log" Jul 17 14:45:33.387923 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:33.387890 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-tp54t_ac4f2a5b-43bb-4e14-a62c-f8a19e93062e/kube-auth-proxy/0.log" Jul 17 14:45:34.015827 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:34.015793 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77668bc899-kmt6q_65d7285f-6dec-477a-b2f4-bfc0989f2ea5/manager/0.log" Jul 17 14:45:34.962622 ip-10-0-133-123 kubenswrapper[2555]: I0717 14:45:34.962588 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-tp54t_ac4f2a5b-43bb-4e14-a62c-f8a19e93062e/kube-auth-proxy/0.log"