Jul 09 22:08:21.576134 ip-10-0-130-127 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 22:08:21.576149 ip-10-0-130-127 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 22:08:21.576157 ip-10-0-130-127 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 22:08:21.576564 ip-10-0-130-127 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 22:08:31.817673 ip-10-0-130-127 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 22:08:31.817697 ip-10-0-130-127 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 16d0fcd5d4f44df488d0c1d4f68045b8 -- Jul 09 22:10:30.158346 ip-10-0-130-127 systemd[1]: Starting Kubernetes Kubelet... Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.555237 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559583 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559594 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559598 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559601 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559606 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559609 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559612 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.577538 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559615 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559618 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559621 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559624 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559627 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559630 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559632 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559635 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559637 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559640 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559643 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559646 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559649 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559654 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559658 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559662 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559666 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559672 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559706 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559710 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.578443 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559714 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559717 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559720 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559723 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559726 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559729 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559733 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559736 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559739 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559743 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559746 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559749 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559752 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559754 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559757 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559760 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559762 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559765 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559768 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559770 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.579726 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559773 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559776 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559778 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559781 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559784 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559786 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559789 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559791 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559793 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559796 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559799 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559802 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559805 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559809 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559811 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559815 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559818 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559820 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559823 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559825 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.580592 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559828 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559831 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559833 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559835 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559839 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559841 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559844 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559847 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559849 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559853 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559855 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559858 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559860 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559863 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559865 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559868 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559870 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559873 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559876 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.581517 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559878 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559881 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559884 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559886 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559889 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559892 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559894 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559897 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559901 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559905 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559908 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559911 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.559914 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560670 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560680 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560683 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560686 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560690 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560693 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.582784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560697 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560699 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560702 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560704 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560707 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560709 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560712 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560714 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560717 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560720 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560722 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560727 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560731 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560734 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560737 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560739 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560744 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560746 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560749 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560753 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.583673 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560755 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560758 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560761 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560763 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560766 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560769 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560773 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560776 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560778 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560781 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560784 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560786 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560789 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560792 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560794 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560797 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560799 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560802 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560804 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560807 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.584531 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560809 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560812 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560814 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560817 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560819 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560821 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560824 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560828 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560832 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560834 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560837 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560840 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560843 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560845 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560848 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560850 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560853 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560855 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560858 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560861 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.585769 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560864 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560866 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560869 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560872 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560874 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560877 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560880 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560883 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560885 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560888 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560890 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560893 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560895 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560897 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560900 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560902 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560904 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560907 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560911 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.586550 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560915 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560918 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560921 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560924 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560927 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560929 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560932 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560935 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560937 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560940 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560942 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560945 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560947 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.560950 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561325 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561330 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561333 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561335 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561338 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561341 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.587358 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561344 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561346 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561354 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561356 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561359 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561362 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561364 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561367 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561370 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561372 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561375 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561377 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561380 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561383 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561385 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561388 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561390 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561392 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561395 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561397 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.588201 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561400 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561402 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561406 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561410 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561413 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561416 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561418 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561421 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561425 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561428 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561431 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561434 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561436 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561439 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561442 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561445 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561447 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561450 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561453 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.589461 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561455 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561457 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561460 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561463 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561465 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561469 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561472 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561474 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561477 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561479 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561482 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561484 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561487 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561489 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561492 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561494 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561497 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561499 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561502 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561505 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.590393 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561509 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561513 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561516 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561519 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561521 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561524 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561527 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561531 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561533 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561536 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561538 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561541 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561543 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561546 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561548 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561550 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561553 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561557 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561559 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561562 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.591729 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561564 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561567 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561569 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561571 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561574 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561577 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561580 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561583 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561585 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561588 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561590 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561592 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561595 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.561597 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561658 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561665 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561671 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561675 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561679 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561691 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561695 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 22:10:30.592521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561699 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561702 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561705 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561708 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561711 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561714 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561717 2575 flags.go:64] FLAG: --cgroup-root="" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561720 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561724 2575 flags.go:64] FLAG: --client-ca-file="" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561727 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561730 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561734 2575 flags.go:64] FLAG: --cluster-domain="" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561737 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561740 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561743 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561746 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561750 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561753 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561756 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561759 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561762 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561765 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561767 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561770 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 22:10:30.593337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561773 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561777 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561780 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561783 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561785 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561788 2575 flags.go:64] FLAG: --enable-server="true" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561793 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561797 2575 flags.go:64] FLAG: --event-burst="100" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561800 2575 flags.go:64] FLAG: --event-qps="50" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561803 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561806 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561809 2575 flags.go:64] FLAG: --eviction-hard="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561812 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561815 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561818 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561821 2575 flags.go:64] FLAG: --eviction-soft="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561825 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561828 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561831 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561834 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561836 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561839 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561842 2575 flags.go:64] FLAG: --feature-gates="" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561846 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561849 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 22:10:30.594134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561851 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561854 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561857 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561860 2575 flags.go:64] FLAG: --help="false" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561863 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-130-127.ec2.internal" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561866 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561869 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561872 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561875 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561878 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561881 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561883 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561886 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561891 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561894 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561897 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561900 2575 flags.go:64] FLAG: --kube-reserved="" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561903 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561905 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561909 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561911 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561914 2575 flags.go:64] FLAG: --lock-file="" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561916 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561920 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 22:10:30.594914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561923 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561928 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561931 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561934 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561937 2575 flags.go:64] FLAG: --logging-format="text" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561939 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561943 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561945 2575 flags.go:64] FLAG: --manifest-url="" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561948 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561955 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561958 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561962 2575 flags.go:64] FLAG: --max-pods="110" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561964 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561967 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561970 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561973 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561976 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561979 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561982 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561988 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561991 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561995 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.561998 2575 flags.go:64] FLAG: --pod-cidr="" Jul 09 22:10:30.602262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562001 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562007 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562010 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562013 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562015 2575 flags.go:64] FLAG: --port="10250" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562018 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562021 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08883808832cd9ff5" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562024 2575 flags.go:64] FLAG: --qos-reserved="" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562027 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562031 2575 flags.go:64] FLAG: --register-node="true" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562033 2575 flags.go:64] FLAG: --register-with-taints="" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562037 2575 flags.go:64] FLAG: --registry-burst="10" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562040 2575 flags.go:64] FLAG: --registry-qps="5" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562042 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562045 2575 flags.go:64] FLAG: --reserved-memory="" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562048 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562051 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562054 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562057 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562060 2575 flags.go:64] FLAG: --runonce="false" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562063 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562079 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562083 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562086 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562088 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562091 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 22:10:30.603055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562094 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562097 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562099 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562102 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562106 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562109 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562112 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562115 2575 flags.go:64] FLAG: --system-cgroups="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562118 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562123 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562126 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562129 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562132 2575 flags.go:64] FLAG: --tls-min-version="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562135 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562138 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562142 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562144 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562147 2575 flags.go:64] FLAG: --v="2" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562151 2575 flags.go:64] FLAG: --version="false" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562154 2575 flags.go:64] FLAG: --vmodule="" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562158 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.562161 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562268 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562271 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.708889 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562274 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562277 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562280 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562283 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562286 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562289 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562291 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562294 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562297 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562299 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562302 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562304 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562308 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562311 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562315 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562317 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562320 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562323 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562325 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562328 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.709688 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562332 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562335 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562338 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562343 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562346 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562348 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562351 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562353 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562356 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562359 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562362 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562366 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562370 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562374 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562377 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562379 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562382 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562385 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562387 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.710474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562390 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562392 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562395 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562397 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562399 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562403 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562406 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562408 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562411 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562413 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562416 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562419 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562421 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562424 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562426 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562429 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562432 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562435 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562438 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562440 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.711265 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562443 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562445 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562448 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562451 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562453 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562456 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562458 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562461 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562463 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562466 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562469 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562471 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562474 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562477 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562479 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562482 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562484 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562488 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562490 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562493 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.712037 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562495 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562498 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562500 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562504 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562506 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562509 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562511 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562513 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562517 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562519 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562522 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562525 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562527 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562530 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562532 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562535 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562537 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.712840 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.562540 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.563257 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.579119 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.579143 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.580146 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.580156 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580257 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580264 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580273 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580279 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580283 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580289 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580299 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580304 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.713643 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580309 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580313 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580316 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580325 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580329 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580334 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580339 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580344 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580347 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580356 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580360 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580364 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580368 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580372 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580375 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580384 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580387 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580392 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580396 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580399 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.714437 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580403 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580420 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580425 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580428 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580431 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580440 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580444 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580449 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580452 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580456 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580460 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580468 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580472 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580476 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580480 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580483 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580487 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580494 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580499 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580502 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.715221 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580506 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580511 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580515 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580523 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580527 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580531 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580535 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580540 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580550 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580555 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580559 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580565 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580574 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580579 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580591 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580595 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580599 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580603 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580608 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.716016 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580616 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580620 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580624 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580628 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580632 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580640 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580644 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580648 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580652 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580657 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580665 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580668 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580673 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580677 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580681 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580689 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580692 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580697 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580701 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580704 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.716812 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580708 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580716 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580720 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580724 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580728 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580732 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580736 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580743 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580754 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580759 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580763 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.580766 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.580778 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581010 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581017 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:10:30.717607 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581021 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581028 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581032 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581037 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581041 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581046 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581050 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581053 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581057 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581061 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581080 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581084 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581092 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581097 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581101 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581105 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581109 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581113 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581120 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:10:30.718011 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581126 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581132 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581136 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581140 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581148 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581152 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581156 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581168 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581176 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581180 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581184 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581188 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581191 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581199 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581203 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581207 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581213 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581217 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581224 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581228 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:10:30.718802 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581232 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581236 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581240 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581248 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581252 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581255 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581260 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581264 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581267 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581275 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581279 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581283 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581287 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581290 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581294 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581302 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581308 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581314 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581318 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:10:30.719590 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581321 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581337 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581342 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581347 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581356 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581361 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581365 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581369 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581373 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581384 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581389 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581393 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581397 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581401 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581405 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581412 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581416 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581420 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581424 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581429 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:10:30.720374 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581432 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581440 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581444 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581449 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581453 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581456 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581459 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581467 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581472 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581476 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581480 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581483 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581487 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581494 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581506 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581510 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581514 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581517 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:10:30.721181 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:30.581523 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.581535 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.581717 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.584894 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.586645 2575 server.go:1019] "Starting client certificate rotation" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.586776 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.586842 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.589633 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 22:10:30.721956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:30.595922 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.124750 2575 log.go:25] "Validated CRI v1 runtime API" Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.127117 2575 log.go:25] "Validated CRI v1 image API" Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.127833 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.130191 2575 fs.go:135] Filesystem UUIDs: map[1eb2ca02-e6a1-4c66-8020-3b3c1aff3c88:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9ed7ce9a-30cb-4132-a18b-cf173d18aa71:/dev/nvme0n1p3] Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.130208 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 22:10:32.191552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.131751 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 22:10:32.156517 ip-10-0-130-127 systemd[1]: Started Kubernetes Kubelet. Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.134688 2575 manager.go:216] Machine: {Timestamp:2026-07-09 22:10:32.134186488 +0000 UTC m=+0.825760532 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103582 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2023e0cc599d9dd304deae959a47c0 SystemUUID:ec2023e0-cc59-9d9d-d304-deae959a47c0 BootID:16d0fcd5-d4f4-4df4-88d0-c1d4f68045b8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f6:e1:7f:a1:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f6:e1:7f:a1:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:04:17:11:d6:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.134917 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.135025 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.138304 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.138330 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.139234 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.139243 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.139250 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.139264 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.141315 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.144973 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145093 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145121 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145130 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145140 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145149 2575 kubelet.go:398] "Adding apiserver pod source" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145160 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 22:10:32.193941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145516 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.145533 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.147450 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.148827 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v6lcr" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.149993 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.150021 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151485 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151509 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151517 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151527 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151554 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151567 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151580 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151587 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151593 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151598 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151612 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.151617 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.152399 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.152406 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.155584 2575 server.go:1268] "Started kubelet" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.155691 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.155790 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.155867 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.155966 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.156045 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.156041 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.156483 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v6lcr" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.157644 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.161270 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.162618 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.168395 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.168650 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.171130 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.171906 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.171910 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.171929 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172001 2575 factory.go:55] Registering systemd factory Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172015 2575 factory.go:223] Registration of the systemd container factory successfully Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172003 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172051 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172179 2575 factory.go:153] Registering CRI-O factory Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172193 2575 factory.go:223] Registration of the crio container factory successfully Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172240 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 22:10:32.194711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172261 2575 factory.go:103] Registering Raw factory Jul 09 22:10:32.196682 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.171445 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-127.ec2.internal.18c0be9c9493150b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-127.ec2.internal,UID:ip-10-0-130-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-127.ec2.internal,},FirstTimestamp:2026-07-09 22:10:32.155559179 +0000 UTC m=+0.847133224,LastTimestamp:2026-07-09 22:10:32.155559179 +0000 UTC m=+0.847133224,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-127.ec2.internal,}" Jul 09 22:10:32.196682 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172279 2575 manager.go:1196] Started watching for new ooms in manager Jul 09 22:10:32.196682 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.172555 2575 manager.go:349] Starting recovery of all containers Jul 09 22:10:32.196682 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.179278 2575 manager.go:354] Recovery completed Jul 09 22:10:32.196682 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.185572 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.460930 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.462719 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.462781 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.462806 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.462820 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.463449 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.463458 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.463462 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.463477 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464211 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464236 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464246 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464754 2575 policy_none.go:49] "None policy: Start" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464767 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.464777 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.465793 2575 policy_none.go:47] "Start" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.473455 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-127.ec2.internal\" not found" node="ip-10-0-130-127.ec2.internal" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526249 2575 manager.go:337] "Starting Device Plugin manager" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.526303 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526312 2575 server.go:87] "Starting device plugin registration server" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526529 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526538 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526621 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526701 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.526710 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.527166 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 22:10:32.546012 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.527208 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.730430 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.730440 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.730582 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.731937 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.731965 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.731975 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.731999 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-127.ec2.internal" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:32.750984 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-127.ec2.internal" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.751008 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-127.ec2.internal\": node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:32.825214 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:32.767950 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:33.184032 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.183929 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 22:10:33.184032 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.183974 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:33.184229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.184004 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 22:05:32 +0000 UTC" deadline="2028-02-14 16:08:59.522263513 +0000 UTC" Jul 09 22:10:33.184229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.184056 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14033h58m26.338212095s" Jul 09 22:10:33.185396 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.185379 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 22:10:33.185485 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.185415 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 22:10:33.185485 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.185435 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 22:10:33.185485 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.185469 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 22:10:33.189542 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.189522 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:10:33.191321 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.191300 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:10:33.284409 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.284378 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:33.286516 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.286498 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal"] Jul 09 22:10:33.286590 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.286571 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:33.287438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.287424 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:33.287507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.287453 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:33.287507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.287462 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:33.289007 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.288996 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:33.289129 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289116 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.289165 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289143 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:33.289669 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289653 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:33.289751 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289674 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:33.289751 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289685 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:33.289751 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289653 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:33.289844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289753 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:33.289844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.289763 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:33.291323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.291311 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.291365 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.291334 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:10:33.291997 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.291984 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:10:33.292063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.292006 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:10:33.292063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.292020 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:10:33.310488 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.310473 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-127.ec2.internal\" not found" node="ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.314748 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.314733 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-127.ec2.internal\" not found" node="ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.385033 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.385009 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:33.385131 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.385098 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.385190 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.385128 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.385190 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.385151 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a92842bdfd3fd5ca29ad4095ae200ef-config\") pod \"kube-apiserver-proxy-ip-10-0-130-127.ec2.internal\" (UID: \"7a92842bdfd3fd5ca29ad4095ae200ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485417 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:33.485356 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-127.ec2.internal\" not found" Jul 09 22:10:33.485417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485393 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485415 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a92842bdfd3fd5ca29ad4095ae200ef-config\") pod \"kube-apiserver-proxy-ip-10-0-130-127.ec2.internal\" (UID: \"7a92842bdfd3fd5ca29ad4095ae200ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485450 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e37e2c09e764fff626bb96e468529745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal\" (UID: \"e37e2c09e764fff626bb96e468529745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.485569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.485466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a92842bdfd3fd5ca29ad4095ae200ef-config\") pod \"kube-apiserver-proxy-ip-10-0-130-127.ec2.internal\" (UID: \"7a92842bdfd3fd5ca29ad4095ae200ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.539313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.539291 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:10:33.571732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.571713 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.583900 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.583875 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 22:10:33.585119 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.585108 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.591698 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.591684 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 22:10:33.614809 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.614794 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.619176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.619126 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" Jul 09 22:10:33.726299 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:33.726276 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:10:34.184311 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.184288 2575 apiserver.go:52] "Watching apiserver" Jul 09 22:10:34.184626 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.184313 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 22:10:34.189861 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.189830 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a92842bdfd3fd5ca29ad4095ae200ef.slice/crio-7a9361c15a18060b29a684ce178110f3682768cc773619511936f32a51049dd6 WatchSource:0}: Error finding container 7a9361c15a18060b29a684ce178110f3682768cc773619511936f32a51049dd6: Status 404 returned error can't find the container with id 7a9361c15a18060b29a684ce178110f3682768cc773619511936f32a51049dd6 Jul 09 22:10:34.190503 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.190468 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37e2c09e764fff626bb96e468529745.slice/crio-67c5c4aace57e0ac8690576142338fd2f18ded915982c3cea452c8349321efb9 WatchSource:0}: Error finding container 67c5c4aace57e0ac8690576142338fd2f18ded915982c3cea452c8349321efb9: Status 404 returned error can't find the container with id 67c5c4aace57e0ac8690576142338fd2f18ded915982c3cea452c8349321efb9 Jul 09 22:10:34.193384 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.193363 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 22:10:34.194600 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.194581 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-djj9b","openshift-ovn-kubernetes/ovnkube-node-d6n4b","kube-system/global-pull-secret-syncer-gddff","kube-system/konnectivity-agent-8kktk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5","openshift-cluster-node-tuning-operator/tuned-t7nhl","openshift-image-registry/node-ca-sh9k4","openshift-multus/multus-additional-cni-plugins-jsk8z","openshift-multus/network-metrics-daemon-pwcd7","openshift-network-operator/iptables-alerter-sjg6v","kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal","openshift-dns/node-resolver-7r2hf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal","openshift-multus/multus-pzjrp"] Jul 09 22:10:34.195438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.195418 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:10:34.196449 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.196432 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 22:10:34.196794 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.196776 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.198185 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.198151 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.200301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.199834 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.200301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.200001 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6rjwr\"" Jul 09 22:10:34.200301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.200117 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 22:10:34.200301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.200249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.200545 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.200320 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:34.200545 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.200514 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.201562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.200712 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.201562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.201158 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 22:10:34.201562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.201245 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.201562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.201421 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 22:10:34.201789 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.201673 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 22:10:34.201789 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.201762 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s55gs\"" Jul 09 22:10:34.202369 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.202034 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.203309 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.203289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.204723 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.204707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.205088 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.204959 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 22:10:34.205088 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.205053 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 22:10:34.205276 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.205262 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vfwm7\"" Jul 09 22:10:34.205625 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.205609 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 22:10:34.205759 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.205741 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.205802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.205795 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.206131 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.206115 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vx9jf\"" Jul 09 22:10:34.206209 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.206188 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.206281 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.206259 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:34.207101 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.207081 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.207330 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.207314 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.207567 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.207547 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:34.207665 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.207600 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b5fbb\"" Jul 09 22:10:34.207665 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.207623 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:34.209114 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.209095 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.210390 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.210371 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.211951 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.211936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.212118 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.212059 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dr7vx\"" Jul 09 22:10:34.212118 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.212068 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 22:10:34.212990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.212925 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.212990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.212981 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 22:10:34.212990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.212987 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.213176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213066 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 22:10:34.213176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213078 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hwhtg\"" Jul 09 22:10:34.213176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213112 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.213176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213172 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.213393 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213205 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 22:10:34.213393 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213364 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 22:10:34.213542 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.213485 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.214275 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.214256 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8l6qk\"" Jul 09 22:10:34.214275 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.214261 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 22:10:34.217133 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.217113 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 22:10:34.217227 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.217133 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-prqwd\"" Jul 09 22:10:34.217227 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.217118 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 22:10:34.220740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.220723 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cct4w" Jul 09 22:10:34.228805 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.228790 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cct4w" Jul 09 22:10:34.273607 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.273592 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 22:10:34.289353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcz9q\" (UniqueName: \"kubernetes.io/projected/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-kube-api-access-lcz9q\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.289408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289358 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.289408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-log-socket\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.289408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cnibin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.289408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-tmp\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.289589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289449 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-env-overrides\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.289589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-k8s-cni-cncf-io\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.289589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-socket-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.289589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-systemd\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.289589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289564 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-netns\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289587 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8gkn\" (UniqueName: \"kubernetes.io/projected/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-kube-api-access-s8gkn\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-registration-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289667 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysconfig\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289720 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trct9\" (UniqueName: \"kubernetes.io/projected/a4af579e-a949-4cc2-942a-6b5560ddf2a4-kube-api-access-trct9\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-kubelet\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289755 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:34.289792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-iptables-alerter-script\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289799 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/075e707e-55d1-476f-b14c-4875d35f240d-hosts-file\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289815 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69jjh\" (UniqueName: \"kubernetes.io/projected/19060103-1510-4fe4-a5da-a7dab5bd4749-kube-api-access-69jjh\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289829 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-sys\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-daemon-config\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-run\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289871 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-tuned\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-slash\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-var-lib-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/715df68e-6d7c-4334-8dc1-f8684bf5a1af-agent-certs\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289973 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-systemd-units\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.289987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-ovn\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/986ae43d-bda5-4c43-a126-3217ec5151a9-ovn-node-metrics-cert\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290016 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-socket-dir-parent\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-conf-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.290117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290059 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/715df68e-6d7c-4334-8dc1-f8684bf5a1af-konnectivity-ca\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290095 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-host-slash\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-bin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwjdp\" (UniqueName: \"kubernetes.io/projected/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kube-api-access-lwjdp\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a3da10f-2ff6-4343-9843-cd58a6fd7339-host\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-var-lib-kubelet\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290198 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-kubelet-config\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e707e-55d1-476f-b14c-4875d35f240d-tmp-dir\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a3da10f-2ff6-4343-9843-cd58a6fd7339-serviceca\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9lj9\" (UniqueName: \"kubernetes.io/projected/5a3da10f-2ff6-4343-9843-cd58a6fd7339-kube-api-access-c9lj9\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290271 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-system-cni-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-cnibin\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-multus\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290356 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.290578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290369 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-kubernetes\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290393 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-systemd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-netd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-etc-kubernetes\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbkvl\" (UniqueName: \"kubernetes.io/projected/4361956b-0edf-45fb-90e6-ab52b1b3286b-kube-api-access-dbkvl\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290469 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7vw\" (UniqueName: \"kubernetes.io/projected/075e707e-55d1-476f-b14c-4875d35f240d-kube-api-access-ht7vw\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290489 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-os-release\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290505 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-lib-modules\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290517 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-host\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-bin\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-config\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290601 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vch\" (UniqueName: \"kubernetes.io/projected/986ae43d-bda5-4c43-a126-3217ec5151a9-kube-api-access-n7vch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-etc-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290633 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-conf\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290718 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-netns\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290739 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-kubelet\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290753 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-hostroot\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-multus-certs\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290780 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-modprobe-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290830 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-node-log\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-script-lib\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-system-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290920 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cni-binary-copy\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290942 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-dbus\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-device-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.290985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-sys-fs\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.291008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.291029 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.291741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.291053 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-os-release\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.391590 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69jjh\" (UniqueName: \"kubernetes.io/projected/19060103-1510-4fe4-a5da-a7dab5bd4749-kube-api-access-69jjh\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.391590 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-sys\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.391590 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-daemon-config\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.391732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391642 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-sys\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.391791 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-run\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.391844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391810 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-tuned\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.391844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391836 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-slash\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.391968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391882 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-run\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.391968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391884 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-slash\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.391968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391921 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-var-lib-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.391968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391961 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391960 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-var-lib-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.391992 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/715df68e-6d7c-4334-8dc1-f8684bf5a1af-agent-certs\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-systemd-units\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392038 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-ovn\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/986ae43d-bda5-4c43-a126-3217ec5151a9-ovn-node-metrics-cert\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392091 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-systemd-units\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392086 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-daemon-config\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-socket-dir-parent\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392142 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-conf-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392089 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/715df68e-6d7c-4334-8dc1-f8684bf5a1af-konnectivity-ca\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392106 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-ovn\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-host-slash\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392195 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-socket-dir-parent\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-bin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwjdp\" (UniqueName: \"kubernetes.io/projected/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kube-api-access-lwjdp\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392242 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-conf-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392252 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-host-slash\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392301 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-bin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a3da10f-2ff6-4343-9843-cd58a6fd7339-host\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-var-lib-kubelet\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392381 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-kubelet-config\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e707e-55d1-476f-b14c-4875d35f240d-tmp-dir\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392427 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a3da10f-2ff6-4343-9843-cd58a6fd7339-serviceca\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392455 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-var-lib-kubelet\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-kubelet-config\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.392602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392513 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a3da10f-2ff6-4343-9843-cd58a6fd7339-host\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9lj9\" (UniqueName: \"kubernetes.io/projected/5a3da10f-2ff6-4343-9843-cd58a6fd7339-kube-api-access-c9lj9\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392945 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-system-cni-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-cnibin\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392736 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/715df68e-6d7c-4334-8dc1-f8684bf5a1af-konnectivity-ca\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392873 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a3da10f-2ff6-4343-9843-cd58a6fd7339-serviceca\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393082 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-multus\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-cni-multus\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.392736 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e707e-55d1-476f-b14c-4875d35f240d-tmp-dir\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393107 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-system-cni-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393051 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393051 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-cnibin\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393133 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-kubernetes\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.393227 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:34.393413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-kubernetes\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.393282 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:34.893264052 +0000 UTC m=+3.584838099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393297 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-systemd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393326 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-netd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393348 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393372 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-etc-kubernetes\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393386 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-systemd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbkvl\" (UniqueName: \"kubernetes.io/projected/4361956b-0edf-45fb-90e6-ab52b1b3286b-kube-api-access-dbkvl\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7vw\" (UniqueName: \"kubernetes.io/projected/075e707e-55d1-476f-b14c-4875d35f240d-kube-api-access-ht7vw\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-os-release\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393458 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-multus-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393492 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-lib-modules\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-host\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393532 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-netd\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-bin\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393550 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-etc-kubernetes\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-config\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vch\" (UniqueName: \"kubernetes.io/projected/986ae43d-bda5-4c43-a126-3217ec5151a9-kube-api-access-n7vch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393641 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-lib-modules\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393650 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-etc-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393694 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-conf\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-netns\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-kubelet\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-hostroot\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-multus-certs\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393827 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-os-release\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393871 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-modprobe-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393878 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-cni-bin\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-node-log\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-script-lib\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.394888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393938 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-host\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.393973 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-var-lib-kubelet\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-system-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394009 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-modprobe-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cni-binary-copy\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-dbus\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394075 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-hostroot\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394084 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-multus-certs\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394089 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-device-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-sys-fs\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394120 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-etc-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394166 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394170 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-etc-selinux\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-os-release\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394212 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcz9q\" (UniqueName: \"kubernetes.io/projected/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-kube-api-access-lcz9q\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.395611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394254 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-conf\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394258 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-log-socket\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394287 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-netns\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cnibin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394308 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-system-cni-dir\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394329 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-tmp\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394333 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cnibin\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394341 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-log-socket\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394370 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-run-openvswitch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394382 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-device-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-env-overrides\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394436 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-os-release\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394443 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-k8s-cni-cncf-io\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.394451 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-socket-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394482 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-sys-fs\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.394492 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:10:34.894477469 +0000 UTC m=+3.586051517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-systemd\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.396329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-netns\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394577 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-socket-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8gkn\" (UniqueName: \"kubernetes.io/projected/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-kube-api-access-s8gkn\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394698 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-node-log\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394755 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-config\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394820 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19060103-1510-4fe4-a5da-a7dab5bd4749-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394831 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-systemd\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394863 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0a8756ff-0a32-454c-afdb-79d5ee1eb974-dbus\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394869 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-run-netns\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-registration-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394925 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394956 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysctl-d\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.394927 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-registration-dir\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysconfig\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trct9\" (UniqueName: \"kubernetes.io/projected/a4af579e-a949-4cc2-942a-6b5560ddf2a4-kube-api-access-trct9\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395064 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-kubelet\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395091 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395105 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-iptables-alerter-script\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395136 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-host-run-k8s-cni-cncf-io\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395164 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/075e707e-55d1-476f-b14c-4875d35f240d-hosts-file\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395173 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/986ae43d-bda5-4c43-a126-3217ec5151a9-host-kubelet\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395212 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-sysconfig\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395283 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/075e707e-55d1-476f-b14c-4875d35f240d-hosts-file\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395413 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-ovnkube-script-lib\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395423 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/986ae43d-bda5-4c43-a126-3217ec5151a9-env-overrides\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395664 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/986ae43d-bda5-4c43-a126-3217ec5151a9-ovn-node-metrics-cert\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395803 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/715df68e-6d7c-4334-8dc1-f8684bf5a1af-agent-certs\") pod \"konnectivity-agent-8kktk\" (UID: \"715df68e-6d7c-4334-8dc1-f8684bf5a1af\") " pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.395976 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-cni-binary-copy\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.396018 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19060103-1510-4fe4-a5da-a7dab5bd4749-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.396051 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-etc-tuned\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.396152 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-iptables-alerter-script\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.397510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.397426 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4af579e-a949-4cc2-942a-6b5560ddf2a4-tmp\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.407946 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.407926 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69jjh\" (UniqueName: \"kubernetes.io/projected/19060103-1510-4fe4-a5da-a7dab5bd4749-kube-api-access-69jjh\") pod \"multus-additional-cni-plugins-jsk8z\" (UID: \"19060103-1510-4fe4-a5da-a7dab5bd4749\") " pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.410367 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.410353 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:34.410413 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.410372 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:34.410413 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.410382 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:34.410476 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.410428 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:34.910415748 +0000 UTC m=+3.601989799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:34.419709 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.419687 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trct9\" (UniqueName: \"kubernetes.io/projected/a4af579e-a949-4cc2-942a-6b5560ddf2a4-kube-api-access-trct9\") pod \"tuned-t7nhl\" (UID: \"a4af579e-a949-4cc2-942a-6b5560ddf2a4\") " pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.419962 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.419943 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8gkn\" (UniqueName: \"kubernetes.io/projected/a71c93a8-6ad2-463f-90fa-44e4b9b995fa-kube-api-access-s8gkn\") pod \"iptables-alerter-sjg6v\" (UID: \"a71c93a8-6ad2-463f-90fa-44e4b9b995fa\") " pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.420115 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.420090 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbkvl\" (UniqueName: \"kubernetes.io/projected/4361956b-0edf-45fb-90e6-ab52b1b3286b-kube-api-access-dbkvl\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.421765 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.421738 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcz9q\" (UniqueName: \"kubernetes.io/projected/4cf6d097-b294-4c28-bb31-1c96d50e7e0f-kube-api-access-lcz9q\") pod \"multus-pzjrp\" (UID: \"4cf6d097-b294-4c28-bb31-1c96d50e7e0f\") " pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.421956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.421942 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwjdp\" (UniqueName: \"kubernetes.io/projected/e9d84611-710b-4e64-8ada-bd2d6c13f9b1-kube-api-access-lwjdp\") pod \"aws-ebs-csi-driver-node-hxvx5\" (UID: \"e9d84611-710b-4e64-8ada-bd2d6c13f9b1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.422519 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.422504 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7vw\" (UniqueName: \"kubernetes.io/projected/075e707e-55d1-476f-b14c-4875d35f240d-kube-api-access-ht7vw\") pod \"node-resolver-7r2hf\" (UID: \"075e707e-55d1-476f-b14c-4875d35f240d\") " pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.423000 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.422982 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vch\" (UniqueName: \"kubernetes.io/projected/986ae43d-bda5-4c43-a126-3217ec5151a9-kube-api-access-n7vch\") pod \"ovnkube-node-d6n4b\" (UID: \"986ae43d-bda5-4c43-a126-3217ec5151a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.423444 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.423428 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9lj9\" (UniqueName: \"kubernetes.io/projected/5a3da10f-2ff6-4343-9843-cd58a6fd7339-kube-api-access-c9lj9\") pod \"node-ca-sh9k4\" (UID: \"5a3da10f-2ff6-4343-9843-cd58a6fd7339\") " pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.508784 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.508748 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sh9k4" Jul 09 22:10:34.514917 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.514883 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" Jul 09 22:10:34.516808 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.516784 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3da10f_2ff6_4343_9843_cd58a6fd7339.slice/crio-e327bd975ff8e4bfb5946b35f29e67ec62a03f9a176cca9efa31de572099f09d WatchSource:0}: Error finding container e327bd975ff8e4bfb5946b35f29e67ec62a03f9a176cca9efa31de572099f09d: Status 404 returned error can't find the container with id e327bd975ff8e4bfb5946b35f29e67ec62a03f9a176cca9efa31de572099f09d Jul 09 22:10:34.522181 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.522163 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:34.522420 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.522400 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19060103_1510_4fe4_a5da_a7dab5bd4749.slice/crio-e8814378bcdcbe8333c0185a8a95547041f68a92742168d2c739520a48327cfb WatchSource:0}: Error finding container e8814378bcdcbe8333c0185a8a95547041f68a92742168d2c739520a48327cfb: Status 404 returned error can't find the container with id e8814378bcdcbe8333c0185a8a95547041f68a92742168d2c739520a48327cfb Jul 09 22:10:34.526195 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.526177 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" Jul 09 22:10:34.532106 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.532082 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod715df68e_6d7c_4334_8dc1_f8684bf5a1af.slice/crio-74bab90c90a2c0fa6f4063f42993559c5e323a5cf846e4b5139636f46d17f920 WatchSource:0}: Error finding container 74bab90c90a2c0fa6f4063f42993559c5e323a5cf846e4b5139636f46d17f920: Status 404 returned error can't find the container with id 74bab90c90a2c0fa6f4063f42993559c5e323a5cf846e4b5139636f46d17f920 Jul 09 22:10:34.532377 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.532339 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" Jul 09 22:10:34.534491 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.534420 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d84611_710b_4e64_8ada_bd2d6c13f9b1.slice/crio-d5338525304630de8b56534c3af60739a860e3a701014319d45412c5c0bdcd44 WatchSource:0}: Error finding container d5338525304630de8b56534c3af60739a860e3a701014319d45412c5c0bdcd44: Status 404 returned error can't find the container with id d5338525304630de8b56534c3af60739a860e3a701014319d45412c5c0bdcd44 Jul 09 22:10:34.536753 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.536698 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:10:34.539916 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.539884 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4af579e_a949_4cc2_942a_6b5560ddf2a4.slice/crio-d48922c0f78b5f5564b67f326ebf2e3cee942812f82a868ea636a2dfe8d30605 WatchSource:0}: Error finding container d48922c0f78b5f5564b67f326ebf2e3cee942812f82a868ea636a2dfe8d30605: Status 404 returned error can't find the container with id d48922c0f78b5f5564b67f326ebf2e3cee942812f82a868ea636a2dfe8d30605 Jul 09 22:10:34.542653 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.542639 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjg6v" Jul 09 22:10:34.543997 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.543980 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986ae43d_bda5_4c43_a126_3217ec5151a9.slice/crio-e8962a62a2a80a4183678fe18d6bbdd9236205b6688725482ce60d2f32d8f2f2 WatchSource:0}: Error finding container e8962a62a2a80a4183678fe18d6bbdd9236205b6688725482ce60d2f32d8f2f2: Status 404 returned error can't find the container with id e8962a62a2a80a4183678fe18d6bbdd9236205b6688725482ce60d2f32d8f2f2 Jul 09 22:10:34.547437 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.547421 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pzjrp" Jul 09 22:10:34.553294 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.553279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7r2hf" Jul 09 22:10:34.553765 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.553741 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71c93a8_6ad2_463f_90fa_44e4b9b995fa.slice/crio-c4d067255dbbf0507f119cf83bdafd141add0b543815e8f661adc88318f99ac7 WatchSource:0}: Error finding container c4d067255dbbf0507f119cf83bdafd141add0b543815e8f661adc88318f99ac7: Status 404 returned error can't find the container with id c4d067255dbbf0507f119cf83bdafd141add0b543815e8f661adc88318f99ac7 Jul 09 22:10:34.556580 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.556560 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf6d097_b294_4c28_bb31_1c96d50e7e0f.slice/crio-3c65dc6cebb0088535dfe8325f04eedb9a111b58364905c03d986fd6c4d1aa7c WatchSource:0}: Error finding container 3c65dc6cebb0088535dfe8325f04eedb9a111b58364905c03d986fd6c4d1aa7c: Status 404 returned error can't find the container with id 3c65dc6cebb0088535dfe8325f04eedb9a111b58364905c03d986fd6c4d1aa7c Jul 09 22:10:34.561941 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:10:34.561907 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075e707e_55d1_476f_b14c_4875d35f240d.slice/crio-a986fc86ce2b9adcca3b9463c1b9f16e17dfd08f84d95076d330e5be65d13519 WatchSource:0}: Error finding container a986fc86ce2b9adcca3b9463c1b9f16e17dfd08f84d95076d330e5be65d13519: Status 404 returned error can't find the container with id a986fc86ce2b9adcca3b9463c1b9f16e17dfd08f84d95076d330e5be65d13519 Jul 09 22:10:34.898208 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.898171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:34.898386 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.898241 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:34.898386 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.898342 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:34.898386 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.898375 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:34.898540 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.898419 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:35.898397279 +0000 UTC m=+4.589971312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:34.898540 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.898443 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:10:35.898430885 +0000 UTC m=+4.590004919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:34.998830 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:34.998796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:34.999061 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.999002 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:34.999061 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.999032 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:34.999061 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.999046 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:34.999236 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:34.999112 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:35.999092799 +0000 UTC m=+4.690666835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:35.206438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.206291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7r2hf" event={"ID":"075e707e-55d1-476f-b14c-4875d35f240d","Type":"ContainerStarted","Data":"a986fc86ce2b9adcca3b9463c1b9f16e17dfd08f84d95076d330e5be65d13519"} Jul 09 22:10:35.224244 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.224208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pzjrp" event={"ID":"4cf6d097-b294-4c28-bb31-1c96d50e7e0f","Type":"ContainerStarted","Data":"3c65dc6cebb0088535dfe8325f04eedb9a111b58364905c03d986fd6c4d1aa7c"} Jul 09 22:10:35.231394 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.231335 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:05:34 +0000 UTC" deadline="2028-01-05 19:29:18.335042651 +0000 UTC" Jul 09 22:10:35.231394 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.231388 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13077h18m43.103658255s" Jul 09 22:10:35.240263 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.240233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjg6v" event={"ID":"a71c93a8-6ad2-463f-90fa-44e4b9b995fa","Type":"ContainerStarted","Data":"c4d067255dbbf0507f119cf83bdafd141add0b543815e8f661adc88318f99ac7"} Jul 09 22:10:35.267052 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.267018 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" event={"ID":"a4af579e-a949-4cc2-942a-6b5560ddf2a4","Type":"ContainerStarted","Data":"d48922c0f78b5f5564b67f326ebf2e3cee942812f82a868ea636a2dfe8d30605"} Jul 09 22:10:35.274013 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.273986 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" event={"ID":"e9d84611-710b-4e64-8ada-bd2d6c13f9b1","Type":"ContainerStarted","Data":"d5338525304630de8b56534c3af60739a860e3a701014319d45412c5c0bdcd44"} Jul 09 22:10:35.291064 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.291034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sh9k4" event={"ID":"5a3da10f-2ff6-4343-9843-cd58a6fd7339","Type":"ContainerStarted","Data":"e327bd975ff8e4bfb5946b35f29e67ec62a03f9a176cca9efa31de572099f09d"} Jul 09 22:10:35.317926 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.317627 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"e8962a62a2a80a4183678fe18d6bbdd9236205b6688725482ce60d2f32d8f2f2"} Jul 09 22:10:35.330240 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.330214 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kktk" event={"ID":"715df68e-6d7c-4334-8dc1-f8684bf5a1af","Type":"ContainerStarted","Data":"74bab90c90a2c0fa6f4063f42993559c5e323a5cf846e4b5139636f46d17f920"} Jul 09 22:10:35.335750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.335724 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerStarted","Data":"e8814378bcdcbe8333c0185a8a95547041f68a92742168d2c739520a48327cfb"} Jul 09 22:10:35.342417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.342390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" event={"ID":"e37e2c09e764fff626bb96e468529745","Type":"ContainerStarted","Data":"67c5c4aace57e0ac8690576142338fd2f18ded915982c3cea452c8349321efb9"} Jul 09 22:10:35.350572 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.349455 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" event={"ID":"7a92842bdfd3fd5ca29ad4095ae200ef","Type":"ContainerStarted","Data":"7a9361c15a18060b29a684ce178110f3682768cc773619511936f32a51049dd6"} Jul 09 22:10:35.908408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.908367 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:35.908581 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:35.908442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:35.923337 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:35.923203 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:35.923337 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:35.923291 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:37.923268438 +0000 UTC m=+6.614842489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:35.924190 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:35.924167 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:35.924303 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:35.924234 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:10:37.924218397 +0000 UTC m=+6.615792431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:36.010149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.010116 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:36.010300 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.010264 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:36.010300 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.010284 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:36.010300 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.010295 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:36.010461 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.010353 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:38.010333883 +0000 UTC m=+6.701907937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:36.186699 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.186620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:36.186858 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.186740 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:36.187159 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.187136 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:36.187259 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.187235 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:36.187321 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.187304 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:36.187463 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:36.187440 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:36.232226 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.232140 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:05:34 +0000 UTC" deadline="2028-04-16 09:06:41.823817042 +0000 UTC" Jul 09 22:10:36.232226 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:36.232195 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15514h56m5.591625966s" Jul 09 22:10:37.929465 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:37.929422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:37.929906 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:37.929519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:37.929906 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:37.929585 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:37.929906 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:37.929627 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:37.929906 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:37.929675 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:10:41.929645257 +0000 UTC m=+10.621219311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:37.929906 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:37.929696 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:41.929686267 +0000 UTC m=+10.621260302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:38.029922 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:38.029869 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:38.030109 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.030084 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:38.030109 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.030109 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:38.030224 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.030123 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:38.030224 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.030190 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:42.030171044 +0000 UTC m=+10.721745095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:38.186680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:38.186601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:38.186680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:38.186645 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:38.187054 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.186758 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:38.187054 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.186823 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:38.187054 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:38.186875 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:38.187054 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:38.186968 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:40.186601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:40.186735 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:40.187103 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:40.187188 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:40.187231 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:40.187338 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:40.187300 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:41.962096 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:41.961879 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:41.962096 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:41.961973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:41.962096 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:41.962033 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:41.962096 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:41.962049 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:41.962096 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:41.962095 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:10:49.962077633 +0000 UTC m=+18.653651680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:41.962775 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:41.962114 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:49.962104477 +0000 UTC m=+18.653678510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:42.063218 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:42.063168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:42.063403 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.063388 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:42.063467 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.063410 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:42.063467 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.063424 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:42.063585 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.063488 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:10:50.063468939 +0000 UTC m=+18.755042987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:42.186530 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:42.186497 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:42.186728 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.186691 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:42.186834 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:42.186787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:42.186924 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.186873 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:42.186989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:42.186953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:42.187074 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:42.187045 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:44.186887 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:44.186703 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:44.186887 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:44.186711 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:44.186887 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:44.186827 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:44.186887 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:44.186854 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:44.187623 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:44.186971 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:44.187623 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:44.187067 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:46.186031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:46.185996 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:46.186031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:46.186019 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:46.186031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:46.186029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:46.186591 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:46.186122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:46.186591 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:46.186218 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:46.186591 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:46.186296 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:48.186275 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:48.186243 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:48.186684 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:48.186345 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:48.186684 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:48.186360 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:48.186684 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:48.186384 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:48.186684 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:48.186469 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:48.186684 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:48.186544 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:50.024673 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.024633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:50.025149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.024694 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:50.025149 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.024818 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:50.025149 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.024878 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:50.025149 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.024906 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:06.02486936 +0000 UTC m=+34.716443399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:10:50.025149 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.024945 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:11:06.024930576 +0000 UTC m=+34.716504612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:10:50.125771 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.125688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:50.125950 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.125841 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:10:50.125950 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.125861 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:10:50.125950 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.125873 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:50.126115 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.125952 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:06.125933713 +0000 UTC m=+34.817507758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:10:50.185853 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.185823 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:50.186020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.185869 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:50.186020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:50.185934 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:50.186020 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.185945 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:50.186184 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.186012 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:50.186184 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:50.186089 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:52.186021 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:52.185989 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:52.186452 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:52.186089 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:52.186452 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:52.186123 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:52.186452 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:52.186198 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:52.186452 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:52.186093 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:52.186452 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:52.186301 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:54.186444 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.186270 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:54.186823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.186270 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:54.186823 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:54.186523 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:54.186823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.186273 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:54.186823 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:54.186594 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:54.186823 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:54.186677 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:54.398423 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.398396 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pzjrp" event={"ID":"4cf6d097-b294-4c28-bb31-1c96d50e7e0f","Type":"ContainerStarted","Data":"c2d138bfcc680de52a34ca79065b6477843ce4360f2f218ff362e87fd4d33c8b"} Jul 09 22:10:54.399800 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.399777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" event={"ID":"a4af579e-a949-4cc2-942a-6b5560ddf2a4","Type":"ContainerStarted","Data":"d52403a1beaa3a9f85edca03e46521817e3934507d6c7e66e7a89533e57e0078"} Jul 09 22:10:54.404103 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.404059 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"3489e68db723dfd5d76978d4c3c8ded39891067709fa092ff0a1121a0a96cd70"} Jul 09 22:10:54.405430 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.405413 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" event={"ID":"7a92842bdfd3fd5ca29ad4095ae200ef","Type":"ContainerStarted","Data":"08173773c01c5dece63889027bd1b21468c83d83a20aaabb1231b926c6bef327"} Jul 09 22:10:54.418201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.418154 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pzjrp" podStartSLOduration=3.091538443 podStartE2EDuration="22.418138678s" totalImagesPullingTime="19.326600235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:54.417135837 +0000 UTC m=+23.108709893" watchObservedRunningTime="2026-07-09 22:10:54.418138678 +0000 UTC m=+23.109712730" Jul 09 22:10:54.429511 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:54.429423 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-127.ec2.internal" podStartSLOduration=21.429406917 podStartE2EDuration="21.429406917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:54.429288147 +0000 UTC m=+23.120862200" watchObservedRunningTime="2026-07-09 22:10:54.429406917 +0000 UTC m=+23.120980970" Jul 09 22:10:55.409201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.409009 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7r2hf" event={"ID":"075e707e-55d1-476f-b14c-4875d35f240d","Type":"ContainerStarted","Data":"cf57c5f7835fb1982023fe36b18b191752093f62f1c781ff59307bffcd139947"} Jul 09 22:10:55.410391 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.410358 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjg6v" event={"ID":"a71c93a8-6ad2-463f-90fa-44e4b9b995fa","Type":"ContainerStarted","Data":"3511ee35af9fa6fc0ff76b169788709815d06d1905c663fb3187869d8bfe6d75"} Jul 09 22:10:55.411676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.411654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" event={"ID":"e9d84611-710b-4e64-8ada-bd2d6c13f9b1","Type":"ContainerStarted","Data":"830550612d5a9d9eabba2d93030a337741211e470e97eb84032e938aa71c4ebd"} Jul 09 22:10:55.413027 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.413003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sh9k4" event={"ID":"5a3da10f-2ff6-4343-9843-cd58a6fd7339","Type":"ContainerStarted","Data":"a5e81c2850ea24a72d66175e198650bd111d34db2bb5c025c0a341dd27e6d28b"} Jul 09 22:10:55.415609 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.415592 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:10:55.415943 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.415927 2575 generic.go:356] "Generic (PLEG): container finished" podID="986ae43d-bda5-4c43-a126-3217ec5151a9" containerID="b84b6d8f1f24f881ba84d0fad5ba9752ff54951246d142b620d81d3846abb001" exitCode=1 Jul 09 22:10:55.416016 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.415984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"d59436eb2b2f67f49b5727f2bdcb587e69a603fe517cd4940d283fac87ffff27"} Jul 09 22:10:55.416057 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.416027 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"7cb3fe1f11cdce553476948097f0eca25133a4d4c597dfc7f0d57f5b7e2c243c"} Jul 09 22:10:55.416096 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.416070 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"6c5f1de3c50e94033f00e8c93a9e4804780c8b0f1cb4aec18e079fa993377c6c"} Jul 09 22:10:55.416128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.416102 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"b773426ff9ff53134b482d9ec7b80acd41668f4c0e1d2dbd0c09f416a3a1dd81"} Jul 09 22:10:55.416165 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.416137 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerDied","Data":"b84b6d8f1f24f881ba84d0fad5ba9752ff54951246d142b620d81d3846abb001"} Jul 09 22:10:55.417287 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.417265 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kktk" event={"ID":"715df68e-6d7c-4334-8dc1-f8684bf5a1af","Type":"ContainerStarted","Data":"3949b7e9d089d91de453029277823af6d5b4b77910e26adae281161ea2eb71b7"} Jul 09 22:10:55.418543 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.418522 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="c904de7762a50ce9cdde97479479af11a108d04bc5e5c6023deadb0dd22f44f4" exitCode=0 Jul 09 22:10:55.418625 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.418582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"c904de7762a50ce9cdde97479479af11a108d04bc5e5c6023deadb0dd22f44f4"} Jul 09 22:10:55.420010 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.419989 2575 generic.go:356] "Generic (PLEG): container finished" podID="e37e2c09e764fff626bb96e468529745" containerID="a7655a6106b454a595cdc2923eb5a639555f60e54aaf4a9c9140f9b70fe5de3a" exitCode=0 Jul 09 22:10:55.420063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.420044 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" event={"ID":"e37e2c09e764fff626bb96e468529745","Type":"ContainerDied","Data":"a7655a6106b454a595cdc2923eb5a639555f60e54aaf4a9c9140f9b70fe5de3a"} Jul 09 22:10:55.425296 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.425251 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t7nhl" podStartSLOduration=4.085423979 podStartE2EDuration="23.425238272s" totalImagesPullingTime="19.339814293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:54.446621737 +0000 UTC m=+23.138195791" watchObservedRunningTime="2026-07-09 22:10:55.425238272 +0000 UTC m=+24.116812324" Jul 09 22:10:55.425621 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.425599 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7r2hf" podStartSLOduration=4.140653128 podStartE2EDuration="23.425589891s" totalImagesPullingTime="19.284936763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:55.424467261 +0000 UTC m=+24.116041313" watchObservedRunningTime="2026-07-09 22:10:55.425589891 +0000 UTC m=+24.117163944" Jul 09 22:10:55.437608 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.437574 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sh9k4" podStartSLOduration=4.107327895 podStartE2EDuration="23.437564382s" totalImagesPullingTime="19.330236487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:55.437341826 +0000 UTC m=+24.128915880" watchObservedRunningTime="2026-07-09 22:10:55.437564382 +0000 UTC m=+24.129138434" Jul 09 22:10:55.453242 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.453174 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8kktk" podStartSLOduration=4.137518687 podStartE2EDuration="23.453157348s" totalImagesPullingTime="19.315638661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:55.452725662 +0000 UTC m=+24.144299716" watchObservedRunningTime="2026-07-09 22:10:55.453157348 +0000 UTC m=+24.144731406" Jul 09 22:10:55.507799 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.507762 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sjg6v" podStartSLOduration=4.176494321 podStartE2EDuration="23.507749591s" totalImagesPullingTime="19.33125527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:55.507625364 +0000 UTC m=+24.199199417" watchObservedRunningTime="2026-07-09 22:10:55.507749591 +0000 UTC m=+24.199323684" Jul 09 22:10:55.514056 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.514034 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:55.625123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.625092 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:55.724748 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.724722 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:55.927569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:55.927524 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:56.028837 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.028803 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:56.186020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.185942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:56.186020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.185985 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:56.186020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.185994 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:56.186324 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:56.186056 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:56.186324 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:56.186144 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:56.186324 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:56.186225 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:56.424869 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.424649 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" event={"ID":"e37e2c09e764fff626bb96e468529745","Type":"ContainerStarted","Data":"464b38b3f4ed1cbc8d823d8a4365d8dcfbd1933322e7bae093bf88c8826ade79"} Jul 09 22:10:56.439677 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.439608 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-127.ec2.internal" podStartSLOduration=23.439592644 podStartE2EDuration="23.439592644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:56.43936332 +0000 UTC m=+25.130937374" watchObservedRunningTime="2026-07-09 22:10:56.439592644 +0000 UTC m=+25.131166697" Jul 09 22:10:56.474116 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.474089 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 22:10:56.540463 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.540272 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T22:10:56.47411253Z","UUID":"6e865193-8c16-4f2f-b60e-cfac991e2c2a","Handler":null,"Name":"","Endpoint":""} Jul 09 22:10:56.542113 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.542091 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 22:10:56.542231 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.542130 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 22:10:56.683552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:56.683512 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:57.081930 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:57.081883 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:57.182969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:57.182931 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:10:57.430471 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:57.430449 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:10:57.431027 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:57.430911 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"1ad4d9c49db4625f46dbb18340a1bbf1efdefc3483cfe272c02adfe0755099a5"} Jul 09 22:10:57.432946 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:57.432911 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" event={"ID":"e9d84611-710b-4e64-8ada-bd2d6c13f9b1","Type":"ContainerStarted","Data":"b178e808ef905328980b95527ddd196cac25ec31de900eb593a774e4522b8f65"} Jul 09 22:10:58.186244 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.186216 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:10:58.186438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.186335 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:10:58.186438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.186348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:10:58.186438 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:58.186342 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:10:58.186438 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:58.186421 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:10:58.186628 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:10:58.186491 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:10:58.437283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.437199 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" event={"ID":"e9d84611-710b-4e64-8ada-bd2d6c13f9b1","Type":"ContainerStarted","Data":"a87c8a644b3539aae91523a54d9d1d3655312b89d0575f15053af0b5cf2f0009"} Jul 09 22:10:58.472628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.472582 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hxvx5" podStartSLOduration=5.085515753 podStartE2EDuration="26.472561435s" totalImagesPullingTime="21.387045682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:58.471983882 +0000 UTC m=+27.163557937" watchObservedRunningTime="2026-07-09 22:10:58.472561435 +0000 UTC m=+27.164135490" Jul 09 22:10:58.745418 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.745353 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:58.746026 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:58.746003 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:59.439881 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:59.439770 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:59.440367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:59.440271 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8kktk" Jul 09 22:10:59.506177 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:10:59.506146 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:00.185966 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.185938 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:00.185966 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.185959 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:00.186145 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.185962 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:00.186145 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:00.186041 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:11:00.186223 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:00.186146 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:11:00.186269 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:00.186248 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:11:00.445170 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.445147 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:11:00.445545 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.445524 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"71ed6fd6ab9beba4626b2425c1269cfb11f2a9afe0da704cbbffd014f6d30262"} Jul 09 22:11:00.445820 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.445788 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:00.446019 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.446006 2575 scope.go:117] "RemoveContainer" containerID="b84b6d8f1f24f881ba84d0fad5ba9752ff54951246d142b620d81d3846abb001" Jul 09 22:11:00.447513 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.447491 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="5a374d6ea7d9fcaeeeaaf4b818f24bd83ae310953003a6bdd3d9cb00f9baa03c" exitCode=0 Jul 09 22:11:00.447622 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.447544 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"5a374d6ea7d9fcaeeeaaf4b818f24bd83ae310953003a6bdd3d9cb00f9baa03c"} Jul 09 22:11:00.461672 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:00.461657 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:01.452601 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.452576 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:11:01.453139 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.452906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" event={"ID":"986ae43d-bda5-4c43-a126-3217ec5151a9","Type":"ContainerStarted","Data":"cc2f89325d7ec4ef678a8c784b7892065f45811297d8cf765f9279f1945ae527"} Jul 09 22:11:01.453215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.453170 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:01.453271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.453243 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:01.467243 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.467221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:01.482364 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.482323 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" podStartSLOduration=9.83161659 podStartE2EDuration="29.482310915s" totalImagesPullingTime="19.650694325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:01.480114198 +0000 UTC m=+30.171688250" watchObservedRunningTime="2026-07-09 22:11:01.482310915 +0000 UTC m=+30.173884968" Jul 09 22:11:01.881742 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:01.881556 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:02.186225 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.186191 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:02.186402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.186304 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:02.186402 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.186309 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:11:02.186402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.186322 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:02.186574 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.186424 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:11:02.186574 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.186537 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:11:02.279397 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.279360 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gddff"] Jul 09 22:11:02.282304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.282280 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djj9b"] Jul 09 22:11:02.283059 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.283037 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pwcd7"] Jul 09 22:11:02.457145 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.457116 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="22132ac08208d69e972aa4f1745c4435755268c96199b0363f4d0ee939c1de5d" exitCode=0 Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.457217 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"22132ac08208d69e972aa4f1745c4435755268c96199b0363f4d0ee939c1de5d"} Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.457230 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.457320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.457412 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:02.457432 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.457519 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:11:02.457676 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:02.457597 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:11:04.186350 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:04.186314 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:04.186929 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:04.186417 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:04.186929 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:04.186442 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:04.186929 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:04.186431 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:11:04.186929 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:04.186530 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:11:04.186929 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:04.186617 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:11:04.463927 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:04.463881 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="21a2816d3858696f621fa4096426a0ab115393bfc7982509718bd2bca5031972" exitCode=0 Jul 09 22:11:04.464101 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:04.463941 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"21a2816d3858696f621fa4096426a0ab115393bfc7982509718bd2bca5031972"} Jul 09 22:11:06.046715 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.046679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:06.047130 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.046733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:06.047130 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.046843 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:11:06.047130 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.046853 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:11:06.047130 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.046911 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:11:38.046881631 +0000 UTC m=+66.738455681 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:11:06.047130 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.046926 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret podName:0a8756ff-0a32-454c-afdb-79d5ee1eb974 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:38.046919341 +0000 UTC m=+66.738493371 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret") pod "global-pull-secret-syncer-gddff" (UID: "0a8756ff-0a32-454c-afdb-79d5ee1eb974") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:11:06.147498 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.147459 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:06.147669 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.147641 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:11:06.147669 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.147662 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:11:06.147765 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.147672 2575 projected.go:196] Error preparing data for projected volume kube-api-access-tcmnt for pod openshift-network-diagnostics/network-check-target-djj9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:11:06.147765 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.147725 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt podName:64003de7-a59e-4d36-83a7-c3cf4c147e92 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:38.14771227 +0000 UTC m=+66.839286323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcmnt" (UniqueName: "kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt") pod "network-check-target-djj9b" (UID: "64003de7-a59e-4d36-83a7-c3cf4c147e92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:11:06.186222 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.186196 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:06.186347 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.186286 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:06.186347 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.186293 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djj9b" podUID="64003de7-a59e-4d36-83a7-c3cf4c147e92" Jul 09 22:11:06.186347 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.186294 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:06.186445 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.186376 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gddff" podUID="0a8756ff-0a32-454c-afdb-79d5ee1eb974" Jul 09 22:11:06.186479 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:06.186454 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:11:06.845527 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.845491 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:06.944711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:06.944678 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:07.115431 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.114050 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-127.ec2.internal" event="NodeReady" Jul 09 22:11:07.115431 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.114200 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 22:11:07.160096 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.160070 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b4w94"] Jul 09 22:11:07.181884 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.181860 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d479r"] Jul 09 22:11:07.182035 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.182002 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.184833 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.184812 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 22:11:07.184978 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.184869 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r6x6z\"" Jul 09 22:11:07.184978 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.184807 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 22:11:07.207395 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.207372 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:07.207395 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.207385 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.210415 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210392 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 22:11:07.210565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210430 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sjfh2\"" Jul 09 22:11:07.210565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210391 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 22:11:07.210565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210534 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 22:11:07.210728 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210645 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 22:11:07.210858 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210840 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 22:11:07.210858 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.210856 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r7pcm\"" Jul 09 22:11:07.211307 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.211289 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d479r"] Jul 09 22:11:07.211399 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.211325 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b4w94"] Jul 09 22:11:07.268680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.268652 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:07.356873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.356839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-config-volume\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.356873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.356876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-cert\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.357098 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.356936 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-tmp-dir\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.357098 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.356957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m7jz\" (UniqueName: \"kubernetes.io/projected/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-kube-api-access-2m7jz\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.357098 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.356985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b8vd\" (UniqueName: \"kubernetes.io/projected/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-kube-api-access-2b8vd\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.357209 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.357097 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458041 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458016 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-tmp-dir\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458050 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m7jz\" (UniqueName: \"kubernetes.io/projected/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-kube-api-access-2m7jz\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.458201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b8vd\" (UniqueName: \"kubernetes.io/projected/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-kube-api-access-2b8vd\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-config-volume\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-cert\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.458443 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:07.458235 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:07.458443 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:07.458297 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:07.958276499 +0000 UTC m=+36.649850552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:07.458443 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458403 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-tmp-dir\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.458846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.458810 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-config-volume\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.463034 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.463012 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-cert\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.467313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.467280 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b8vd\" (UniqueName: \"kubernetes.io/projected/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-kube-api-access-2b8vd\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.468631 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.468609 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m7jz\" (UniqueName: \"kubernetes.io/projected/b45a8325-8a9b-41f0-ba02-0b31ddd5aea1-kube-api-access-2m7jz\") pod \"ingress-canary-d479r\" (UID: \"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1\") " pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.522139 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.522112 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d479r" Jul 09 22:11:07.701012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.700981 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d479r"] Jul 09 22:11:07.705018 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:11:07.704983 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45a8325_8a9b_41f0_ba02_0b31ddd5aea1.slice/crio-d73b369b4ab64ce469d40db25cc8c15aed1be9037a288e8093f34e15ad62e264 WatchSource:0}: Error finding container d73b369b4ab64ce469d40db25cc8c15aed1be9037a288e8093f34e15ad62e264: Status 404 returned error can't find the container with id d73b369b4ab64ce469d40db25cc8c15aed1be9037a288e8093f34e15ad62e264 Jul 09 22:11:07.962543 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:07.962472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:07.962682 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:07.962642 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:07.962743 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:07.962710 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:08.962691898 +0000 UTC m=+37.654265929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:08.186575 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.186544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:08.186931 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.186544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:08.189818 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.189800 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 22:11:08.191014 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.190995 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jtrvn\"" Jul 09 22:11:08.191073 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.191021 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 22:11:08.473814 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.473778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d479r" event={"ID":"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1","Type":"ContainerStarted","Data":"d73b369b4ab64ce469d40db25cc8c15aed1be9037a288e8093f34e15ad62e264"} Jul 09 22:11:08.970522 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:08.970420 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:08.970710 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:08.970610 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:08.970710 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:08.970690 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:10.970672015 +0000 UTC m=+39.662246071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:09.915803 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:09.915742 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:09.980912 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:09.980860 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:10.985813 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:10.985785 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:10.986183 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:10.985920 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:10.986183 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:10.985974 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:14.985960433 +0000 UTC m=+43.677534464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:11.483732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.483540 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="50e5b16cf6e3929992df22ba3122fbcf5c8320af3f716e2fc76d33df7496b60c" exitCode=0 Jul 09 22:11:11.483921 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.483613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"50e5b16cf6e3929992df22ba3122fbcf5c8320af3f716e2fc76d33df7496b60c"} Jul 09 22:11:11.485165 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.485141 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d479r" event={"ID":"b45a8325-8a9b-41f0-ba02-0b31ddd5aea1","Type":"ContainerStarted","Data":"e19fb046efeceb045f40063dedcb9f5cf18e0f7089fe50c565070a29745330c1"} Jul 09 22:11:11.522729 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.522678 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d479r" podStartSLOduration=1.18964501 podStartE2EDuration="4.522662449s" totalImagesPullingTime="3.333017439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:11.521563431 +0000 UTC m=+40.213137485" watchObservedRunningTime="2026-07-09 22:11:11.522662449 +0000 UTC m=+40.214236523" Jul 09 22:11:11.676600 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.676577 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:11.733588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:11.733566 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:12.172435 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:12.172400 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:12.227297 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:12.227267 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:12.490406 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:12.490331 2575 generic.go:356] "Generic (PLEG): container finished" podID="19060103-1510-4fe4-a5da-a7dab5bd4749" containerID="3f056bd5075104ab14b2ff31375ab23f5fcf7b298b46c1be142fae0bfee5288e" exitCode=0 Jul 09 22:11:12.490564 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:12.490423 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerDied","Data":"3f056bd5075104ab14b2ff31375ab23f5fcf7b298b46c1be142fae0bfee5288e"} Jul 09 22:11:13.495338 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.495304 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" event={"ID":"19060103-1510-4fe4-a5da-a7dab5bd4749","Type":"ContainerStarted","Data":"622ec683f0610df0043b0e8d32311f0a0da7d8fb55507a692611a5913787d1a3"} Jul 09 22:11:13.519554 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.519500 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jsk8z" podStartSLOduration=8.387799 podStartE2EDuration="41.519484773s" totalImagesPullingTime="33.131685773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:13.517963615 +0000 UTC m=+42.209537667" watchObservedRunningTime="2026-07-09 22:11:13.519484773 +0000 UTC m=+42.211058826" Jul 09 22:11:13.638818 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.638766 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:13.697134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.697103 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:13.840325 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.840236 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:13.894279 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:13.894242 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:15.009945 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:15.009885 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:15.010330 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:15.010059 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:15.010330 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:15.010126 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:23.010110227 +0000 UTC m=+51.701684258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:23.064446 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:23.064387 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:23.064918 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:23.064527 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:23.064918 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:23.064591 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:11:39.064576253 +0000 UTC m=+67.756150285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:30.877848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.877817 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw"] Jul 09 22:11:30.880562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.880547 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:30.884514 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.884496 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 22:11:30.884634 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.884541 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 22:11:30.884634 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.884494 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9vsbn\"" Jul 09 22:11:30.884634 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.884604 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 22:11:30.884634 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.884494 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 22:11:30.891030 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.891012 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw"] Jul 09 22:11:30.917758 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.917736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05b579d7-e249-43a6-a665-a2f8336714aa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:30.917846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:30.917765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qskpg\" (UniqueName: \"kubernetes.io/projected/05b579d7-e249-43a6-a665-a2f8336714aa-kube-api-access-qskpg\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.018141 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.018113 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05b579d7-e249-43a6-a665-a2f8336714aa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.018304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.018146 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qskpg\" (UniqueName: \"kubernetes.io/projected/05b579d7-e249-43a6-a665-a2f8336714aa-kube-api-access-qskpg\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.021602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.021582 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05b579d7-e249-43a6-a665-a2f8336714aa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.026443 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.026421 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qskpg\" (UniqueName: \"kubernetes.io/projected/05b579d7-e249-43a6-a665-a2f8336714aa-kube-api-access-qskpg\") pod \"managed-serviceaccount-addon-agent-647d99664b-qjhfw\" (UID: \"05b579d7-e249-43a6-a665-a2f8336714aa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.201191 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.201166 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" Jul 09 22:11:31.333949 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.333921 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw"] Jul 09 22:11:31.337109 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:11:31.337085 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05b579d7_e249_43a6_a665_a2f8336714aa.slice/crio-d791fa7975d712025470d26e16848d67a7071a412b3ed234e2976faa5b2dc0f5 WatchSource:0}: Error finding container d791fa7975d712025470d26e16848d67a7071a412b3ed234e2976faa5b2dc0f5: Status 404 returned error can't find the container with id d791fa7975d712025470d26e16848d67a7071a412b3ed234e2976faa5b2dc0f5 Jul 09 22:11:31.537779 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:31.537704 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" event={"ID":"05b579d7-e249-43a6-a665-a2f8336714aa","Type":"ContainerStarted","Data":"d791fa7975d712025470d26e16848d67a7071a412b3ed234e2976faa5b2dc0f5"} Jul 09 22:11:33.469954 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:33.469929 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d6n4b" Jul 09 22:11:33.641562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:33.641525 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:33.744485 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:33.744391 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:33.842539 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:33.842513 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:33.943506 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:33.943481 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:11:34.545933 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:34.545848 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" event={"ID":"05b579d7-e249-43a6-a665-a2f8336714aa","Type":"ContainerStarted","Data":"d5825e9540238213ae1464d959d214805dc481d2b99ddbff6f10b5eb0379ad39"} Jul 09 22:11:34.562436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:34.562387 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" podStartSLOduration=1.6734618380000001 podStartE2EDuration="4.562373639s" totalImagesPullingTime="2.888911801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:34.561548406 +0000 UTC m=+63.253122460" watchObservedRunningTime="2026-07-09 22:11:34.562373639 +0000 UTC m=+63.253947691" Jul 09 22:11:38.067304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.067249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:11:38.067712 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.067325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:38.070239 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.070220 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 22:11:38.070466 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.070451 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 22:11:38.077460 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:38.077446 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:11:38.077542 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:38.077500 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:12:42.077484553 +0000 UTC m=+130.769058596 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : secret "metrics-daemon-secret" not found Jul 09 22:11:38.079527 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.079505 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0a8756ff-0a32-454c-afdb-79d5ee1eb974-original-pull-secret\") pod \"global-pull-secret-syncer-gddff\" (UID: \"0a8756ff-0a32-454c-afdb-79d5ee1eb974\") " pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:38.167882 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.167853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:38.171099 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.171082 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 22:11:38.180650 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.180632 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 22:11:38.191106 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.191078 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcmnt\" (UniqueName: \"kubernetes.io/projected/64003de7-a59e-4d36-83a7-c3cf4c147e92-kube-api-access-tcmnt\") pod \"network-check-target-djj9b\" (UID: \"64003de7-a59e-4d36-83a7-c3cf4c147e92\") " pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:38.200973 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.200957 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gddff" Jul 09 22:11:38.312412 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.312383 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gddff"] Jul 09 22:11:38.315111 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:11:38.315085 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8756ff_0a32_454c_afdb_79d5ee1eb974.slice/crio-3a4c98c718b6222705242aea70cc9ad7a01f24de5bc0bee21cb1220b75960a1b WatchSource:0}: Error finding container 3a4c98c718b6222705242aea70cc9ad7a01f24de5bc0bee21cb1220b75960a1b: Status 404 returned error can't find the container with id 3a4c98c718b6222705242aea70cc9ad7a01f24de5bc0bee21cb1220b75960a1b Jul 09 22:11:38.420222 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.420196 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sjfh2\"" Jul 09 22:11:38.428408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.428394 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:38.541720 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.541691 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djj9b"] Jul 09 22:11:38.544784 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:11:38.544761 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64003de7_a59e_4d36_83a7_c3cf4c147e92.slice/crio-a1d7087e21426b08a7b11ef93200668e482096b1e887bc584af86e4b912bb437 WatchSource:0}: Error finding container a1d7087e21426b08a7b11ef93200668e482096b1e887bc584af86e4b912bb437: Status 404 returned error can't find the container with id a1d7087e21426b08a7b11ef93200668e482096b1e887bc584af86e4b912bb437 Jul 09 22:11:38.559008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.558979 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gddff" event={"ID":"0a8756ff-0a32-454c-afdb-79d5ee1eb974","Type":"ContainerStarted","Data":"3a4c98c718b6222705242aea70cc9ad7a01f24de5bc0bee21cb1220b75960a1b"} Jul 09 22:11:38.559939 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:38.559918 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djj9b" event={"ID":"64003de7-a59e-4d36-83a7-c3cf4c147e92","Type":"ContainerStarted","Data":"a1d7087e21426b08a7b11ef93200668e482096b1e887bc584af86e4b912bb437"} Jul 09 22:11:39.071732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:39.071686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:11:39.072205 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:39.071950 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:39.072205 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:11:39.072084 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:11.072046596 +0000 UTC m=+99.763620626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:11:44.576953 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:44.576908 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djj9b" event={"ID":"64003de7-a59e-4d36-83a7-c3cf4c147e92","Type":"ContainerStarted","Data":"8da4f60d679cb67d5c4d5e53ebbecc05b27ae1a343d1aae298af46cb98c7b657"} Jul 09 22:11:44.577379 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:44.576971 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:11:44.578254 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:44.578233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gddff" event={"ID":"0a8756ff-0a32-454c-afdb-79d5ee1eb974","Type":"ContainerStarted","Data":"412a9ff560a4611a9914c4e7fb6cea06112888a0ec6be7a3865ce228d9a77cad"} Jul 09 22:11:44.591907 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:44.591844 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-djj9b" podStartSLOduration=67.535583431 podStartE2EDuration="1m12.591828788s" totalImagesPullingTime="5.056245357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:44.591206723 +0000 UTC m=+73.282780776" watchObservedRunningTime="2026-07-09 22:11:44.591828788 +0000 UTC m=+73.283402842" Jul 09 22:11:44.605560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:11:44.605523 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gddff" podStartSLOduration=67.314326777 podStartE2EDuration="1m12.605510765s" totalImagesPullingTime="5.291183988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:44.605098913 +0000 UTC m=+73.296672966" watchObservedRunningTime="2026-07-09 22:11:44.605510765 +0000 UTC m=+73.297084817" Jul 09 22:12:11.079551 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:12:11.079428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:12:11.080026 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:12:11.079556 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:12:11.080026 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:12:11.079623 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:15.079606782 +0000 UTC m=+163.771180816 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:12:15.583233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:12:15.583205 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-djj9b" Jul 09 22:12:42.079101 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:12:42.079069 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:12:42.079568 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:12:42.079180 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:12:42.079568 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:12:42.079240 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:14:44.079225828 +0000 UTC m=+252.770799859 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : secret "metrics-daemon-secret" not found Jul 09 22:13:10.192847 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:13:10.192802 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b4w94" podUID="b8f0f8e8-a18a-450e-81af-c9385ea3c9a7" Jul 09 22:13:10.775973 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:10.775947 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b4w94" Jul 09 22:13:11.196657 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:13:11.196620 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:13:15.092384 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:15.092340 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:13:15.092979 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:13:15.092511 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:13:15.092979 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:13:15.092591 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls podName:b8f0f8e8-a18a-450e-81af-c9385ea3c9a7 nodeName:}" failed. No retries permitted until 2026-07-09 22:15:17.092570208 +0000 UTC m=+285.784144255 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls") pod "dns-default-b4w94" (UID: "b8f0f8e8-a18a-450e-81af-c9385ea3c9a7") : secret "dns-default-metrics-tls" not found Jul 09 22:13:24.186519 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:24.186444 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:13:34.834135 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:34.834103 2575 generic.go:356] "Generic (PLEG): container finished" podID="05b579d7-e249-43a6-a665-a2f8336714aa" containerID="d5825e9540238213ae1464d959d214805dc481d2b99ddbff6f10b5eb0379ad39" exitCode=255 Jul 09 22:13:34.834488 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:34.834155 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" event={"ID":"05b579d7-e249-43a6-a665-a2f8336714aa","Type":"ContainerDied","Data":"d5825e9540238213ae1464d959d214805dc481d2b99ddbff6f10b5eb0379ad39"} Jul 09 22:13:34.839560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:34.839540 2575 scope.go:117] "RemoveContainer" containerID="d5825e9540238213ae1464d959d214805dc481d2b99ddbff6f10b5eb0379ad39" Jul 09 22:13:35.839255 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:35.839222 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647d99664b-qjhfw" event={"ID":"05b579d7-e249-43a6-a665-a2f8336714aa","Type":"ContainerStarted","Data":"aae9ef6bec05ba7a2c4bac71b44e082d265931c598a1f76699d24dc81ca40671"} Jul 09 22:13:58.203739 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:13:58.203709 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:24.962411 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:24.962375 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:25.069485 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:25.069440 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:30.947032 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.946996 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:14:30.949007 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.948989 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss"] Jul 09 22:14:30.949130 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.949113 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:30.950677 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.950661 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" Jul 09 22:14:30.967528 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.967510 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 22:14:30.968671 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.968654 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 22:14:30.968762 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.968694 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 22:14:30.968908 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.968881 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-r5q68\"" Jul 09 22:14:30.974045 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.974027 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 22:14:30.974131 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.974063 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:14:30.983736 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.983721 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jn276\"" Jul 09 22:14:30.986904 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.986875 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 22:14:30.994464 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.994442 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss"] Jul 09 22:14:30.995305 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:30.995289 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:14:31.077028 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077004 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077115 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q969w\" (UniqueName: \"kubernetes.io/projected/d2f0e289-c0c9-404f-b7a7-60c2a6741163-kube-api-access-q969w\") pod \"volume-data-source-validator-5d456fdbcd-kjjss\" (UID: \"d2f0e289-c0c9-404f-b7a7-60c2a6741163\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" Jul 09 22:14:31.077144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077141 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077181 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077300 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.077329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.077314 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmxm\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.094262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.094233 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-phfph"] Jul 09 22:14:31.096047 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.096032 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lrvcq"] Jul 09 22:14:31.096180 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.096165 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" Jul 09 22:14:31.097368 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.097354 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb"] Jul 09 22:14:31.097471 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.097459 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.098559 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.098542 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kvw57\"" Jul 09 22:14:31.098879 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.098864 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.102378 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.102362 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-654ht\"" Jul 09 22:14:31.102466 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.102457 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 22:14:31.102577 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.102563 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 22:14:31.102842 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.102828 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d8sq5\"" Jul 09 22:14:31.103782 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.103756 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 22:14:31.103879 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.103758 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:14:31.104015 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.103836 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 22:14:31.104487 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.104474 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 22:14:31.114942 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.114924 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 22:14:31.115089 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.115069 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 22:14:31.116550 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.116534 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8btqh"] Jul 09 22:14:31.118360 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.118348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.133374 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.133357 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 22:14:31.133459 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.133385 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 22:14:31.133624 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.133610 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 22:14:31.133767 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.133755 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 22:14:31.134555 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.134542 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mx2l8\"" Jul 09 22:14:31.135304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.135289 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb"] Jul 09 22:14:31.141704 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.141675 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 22:14:31.156658 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.156638 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 22:14:31.168112 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.168092 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8btqh"] Jul 09 22:14:31.176148 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.176128 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-phfph"] Jul 09 22:14:31.177699 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48090bc2-a6c4-4412-826a-cc4c65e24085-serving-cert\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.177776 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jf7d\" (UniqueName: \"kubernetes.io/projected/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-kube-api-access-8jf7d\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.177776 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.177848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177814 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.177888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.177888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-trusted-ca\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.177977 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177922 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.177977 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-snapshots\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.177977 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.177973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q969w\" (UniqueName: \"kubernetes.io/projected/d2f0e289-c0c9-404f-b7a7-60c2a6741163-kube-api-access-q969w\") pod \"volume-data-source-validator-5d456fdbcd-kjjss\" (UID: \"d2f0e289-c0c9-404f-b7a7-60c2a6741163\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" Jul 09 22:14:31.178121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178035 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178110 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.178245 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-tmp\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.178245 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.178336 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.178271 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:14:31.178336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-config\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.178336 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.178288 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776765f57c-9mb8c: secret "image-registry-tls" not found Jul 09 22:14:31.178450 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.178358 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls podName:a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:31.678343785 +0000 UTC m=+240.369917821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls") pod "image-registry-776765f57c-9mb8c" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0") : secret "image-registry-tls" not found Jul 09 22:14:31.178450 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178352 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmxm\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178450 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178416 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-serving-cert\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178475 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfmz\" (UniqueName: \"kubernetes.io/projected/84465f8d-067f-407e-9efb-aeec5b9d16c0-kube-api-access-fbfmz\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178529 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84465f8d-067f-407e-9efb-aeec5b9d16c0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178553 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vp7z\" (UniqueName: \"kubernetes.io/projected/a695a8ee-af45-4222-8ff8-43114a675327-kube-api-access-2vp7z\") pod \"network-check-source-989f8b8dc-phfph\" (UID: \"a695a8ee-af45-4222-8ff8-43114a675327\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178579 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.178661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178582 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prrtl\" (UniqueName: \"kubernetes.io/projected/48090bc2-a6c4-4412-826a-cc4c65e24085-kube-api-access-prrtl\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.179037 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.178946 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.179480 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.179459 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.180282 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.180257 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.180367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.180296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.216022 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.215975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q969w\" (UniqueName: \"kubernetes.io/projected/d2f0e289-c0c9-404f-b7a7-60c2a6741163-kube-api-access-q969w\") pod \"volume-data-source-validator-5d456fdbcd-kjjss\" (UID: \"d2f0e289-c0c9-404f-b7a7-60c2a6741163\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" Jul 09 22:14:31.216104 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.216079 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lrvcq"] Jul 09 22:14:31.217383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.217353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmxm\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.218795 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.218768 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.263630 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.263609 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" Jul 09 22:14:31.278987 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.278967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-serving-cert\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.279083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfmz\" (UniqueName: \"kubernetes.io/projected/84465f8d-067f-407e-9efb-aeec5b9d16c0-kube-api-access-fbfmz\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.279083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84465f8d-067f-407e-9efb-aeec5b9d16c0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.279083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279052 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vp7z\" (UniqueName: \"kubernetes.io/projected/a695a8ee-af45-4222-8ff8-43114a675327-kube-api-access-2vp7z\") pod \"network-check-source-989f8b8dc-phfph\" (UID: \"a695a8ee-af45-4222-8ff8-43114a675327\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" Jul 09 22:14:31.279083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prrtl\" (UniqueName: \"kubernetes.io/projected/48090bc2-a6c4-4412-826a-cc4c65e24085-kube-api-access-prrtl\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279285 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279116 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48090bc2-a6c4-4412-826a-cc4c65e24085-serving-cert\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279285 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279143 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jf7d\" (UniqueName: \"kubernetes.io/projected/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-kube-api-access-8jf7d\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.279285 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279285 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-trusted-ca\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.279285 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-snapshots\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-tmp\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.279526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279395 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.279526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-config\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.279851 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84465f8d-067f-407e-9efb-aeec5b9d16c0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280002 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-tmp\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280037 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48090bc2-a6c4-4412-826a-cc4c65e24085-snapshots\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.280106 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280260 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-trusted-ca\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.280265 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:31.780243506 +0000 UTC m=+240.471817556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:31.280401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.280848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280526 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48090bc2-a6c4-4412-826a-cc4c65e24085-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.280848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.280629 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-config\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.281515 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.281497 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-serving-cert\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.281717 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.281697 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48090bc2-a6c4-4412-826a-cc4c65e24085-serving-cert\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.296012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.295991 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prrtl\" (UniqueName: \"kubernetes.io/projected/48090bc2-a6c4-4412-826a-cc4c65e24085-kube-api-access-prrtl\") pod \"insights-operator-5c847dcfc4-8btqh\" (UID: \"48090bc2-a6c4-4412-826a-cc4c65e24085\") " pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.305550 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.305527 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jf7d\" (UniqueName: \"kubernetes.io/projected/e35b1e39-6bb4-417a-b8a6-a1560095a0c8-kube-api-access-8jf7d\") pod \"console-operator-68b757fb67-lrvcq\" (UID: \"e35b1e39-6bb4-417a-b8a6-a1560095a0c8\") " pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.305952 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.305934 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vp7z\" (UniqueName: \"kubernetes.io/projected/a695a8ee-af45-4222-8ff8-43114a675327-kube-api-access-2vp7z\") pod \"network-check-source-989f8b8dc-phfph\" (UID: \"a695a8ee-af45-4222-8ff8-43114a675327\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" Jul 09 22:14:31.307265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.307224 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfmz\" (UniqueName: \"kubernetes.io/projected/84465f8d-067f-407e-9efb-aeec5b9d16c0-kube-api-access-fbfmz\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.405736 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.405703 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" Jul 09 22:14:31.408314 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.408260 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss"] Jul 09 22:14:31.412553 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.412534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:31.412931 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:31.412911 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f0e289_c0c9_404f_b7a7_60c2a6741163.slice/crio-d9886e481f3e3d544e491c3f407f21a195781a3ada6b3f6f128ef0551c8a1cb3 WatchSource:0}: Error finding container d9886e481f3e3d544e491c3f407f21a195781a3ada6b3f6f128ef0551c8a1cb3: Status 404 returned error can't find the container with id d9886e481f3e3d544e491c3f407f21a195781a3ada6b3f6f128ef0551c8a1cb3 Jul 09 22:14:31.425689 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.425663 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" Jul 09 22:14:31.568121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.568094 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-phfph"] Jul 09 22:14:31.568320 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:31.568301 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda695a8ee_af45_4222_8ff8_43114a675327.slice/crio-73fafd6359e589bd4222029c68f5d322afd351b092d7a5b37bd6545c16e251b1 WatchSource:0}: Error finding container 73fafd6359e589bd4222029c68f5d322afd351b092d7a5b37bd6545c16e251b1: Status 404 returned error can't find the container with id 73fafd6359e589bd4222029c68f5d322afd351b092d7a5b37bd6545c16e251b1 Jul 09 22:14:31.604494 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.604458 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lrvcq"] Jul 09 22:14:31.605110 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.605093 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-8btqh"] Jul 09 22:14:31.605596 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:31.605574 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35b1e39_6bb4_417a_b8a6_a1560095a0c8.slice/crio-e8e9a9af175bff3e4fc13c1a4b0a43a55b1749d960a353d2468d1643f75ae887 WatchSource:0}: Error finding container e8e9a9af175bff3e4fc13c1a4b0a43a55b1749d960a353d2468d1643f75ae887: Status 404 returned error can't find the container with id e8e9a9af175bff3e4fc13c1a4b0a43a55b1749d960a353d2468d1643f75ae887 Jul 09 22:14:31.606697 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:31.606641 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48090bc2_a6c4_4412_826a_cc4c65e24085.slice/crio-3315b7fb192aa422c1b76e02b15575b68bd1a6e2ecee3ab2b1b5e627a5669a37 WatchSource:0}: Error finding container 3315b7fb192aa422c1b76e02b15575b68bd1a6e2ecee3ab2b1b5e627a5669a37: Status 404 returned error can't find the container with id 3315b7fb192aa422c1b76e02b15575b68bd1a6e2ecee3ab2b1b5e627a5669a37 Jul 09 22:14:31.649600 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.649570 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:31.683082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.683059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:31.683208 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.683194 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:14:31.683246 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.683209 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776765f57c-9mb8c: secret "image-registry-tls" not found Jul 09 22:14:31.683279 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.683257 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls podName:a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:32.683241989 +0000 UTC m=+241.374816037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls") pod "image-registry-776765f57c-9mb8c" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0") : secret "image-registry-tls" not found Jul 09 22:14:31.784144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.784077 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:31.784259 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.784223 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:31.784298 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:31.784287 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:32.784272602 +0000 UTC m=+241.475846634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:31.805843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.805813 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:31.995618 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.995584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" event={"ID":"d2f0e289-c0c9-404f-b7a7-60c2a6741163","Type":"ContainerStarted","Data":"d9886e481f3e3d544e491c3f407f21a195781a3ada6b3f6f128ef0551c8a1cb3"} Jul 09 22:14:31.996703 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.996683 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" event={"ID":"e35b1e39-6bb4-417a-b8a6-a1560095a0c8","Type":"ContainerStarted","Data":"e8e9a9af175bff3e4fc13c1a4b0a43a55b1749d960a353d2468d1643f75ae887"} Jul 09 22:14:31.997734 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.997712 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" event={"ID":"48090bc2-a6c4-4412-826a-cc4c65e24085","Type":"ContainerStarted","Data":"3315b7fb192aa422c1b76e02b15575b68bd1a6e2ecee3ab2b1b5e627a5669a37"} Jul 09 22:14:31.999147 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.999127 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" event={"ID":"a695a8ee-af45-4222-8ff8-43114a675327","Type":"ContainerStarted","Data":"75b84bc4585f96b151c8c70e356851bfd6c7775b94bc41ddb5c168d81af285db"} Jul 09 22:14:31.999230 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:31.999160 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" event={"ID":"a695a8ee-af45-4222-8ff8-43114a675327","Type":"ContainerStarted","Data":"73fafd6359e589bd4222029c68f5d322afd351b092d7a5b37bd6545c16e251b1"} Jul 09 22:14:32.063379 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:32.063275 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-phfph" podStartSLOduration=1.063254255 podStartE2EDuration="1.063254255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:32.06193769 +0000 UTC m=+240.753511766" watchObservedRunningTime="2026-07-09 22:14:32.063254255 +0000 UTC m=+240.754828313" Jul 09 22:14:32.690937 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:32.690907 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:32.691051 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:32.691013 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:14:32.691051 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:32.691028 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776765f57c-9mb8c: secret "image-registry-tls" not found Jul 09 22:14:32.691149 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:32.691085 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls podName:a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:34.691067742 +0000 UTC m=+243.382641783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls") pod "image-registry-776765f57c-9mb8c" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0") : secret "image-registry-tls" not found Jul 09 22:14:32.792347 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:32.792320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:32.792506 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:32.792469 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:32.792573 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:32.792526 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:34.792507804 +0000 UTC m=+243.484081839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:33.004497 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:33.004410 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" event={"ID":"d2f0e289-c0c9-404f-b7a7-60c2a6741163","Type":"ContainerStarted","Data":"b3b73751029faccc830665a718b42c0faf496840fb536fcc2552c8dac133cdd1"} Jul 09 22:14:33.032711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:33.032665 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kjjss" podStartSLOduration=1.761364851 podStartE2EDuration="3.032648048s" totalImagesPullingTime="1.271283197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:33.032376808 +0000 UTC m=+241.723950862" watchObservedRunningTime="2026-07-09 22:14:33.032648048 +0000 UTC m=+241.724222102" Jul 09 22:14:33.173127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:33.173082 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:33.297420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:33.297337 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:34.709031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:34.709003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:34.709375 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:34.709158 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:14:34.709375 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:34.709179 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776765f57c-9mb8c: secret "image-registry-tls" not found Jul 09 22:14:34.709375 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:34.709232 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls podName:a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:38.709217022 +0000 UTC m=+247.400791053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls") pod "image-registry-776765f57c-9mb8c" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0") : secret "image-registry-tls" not found Jul 09 22:14:34.810017 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:34.809985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:34.810160 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:34.810130 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:34.810206 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:34.810191 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:38.81017706 +0000 UTC m=+247.501751091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:35.011851 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.011763 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" event={"ID":"48090bc2-a6c4-4412-826a-cc4c65e24085","Type":"ContainerStarted","Data":"52ee52e3628d5be4886cf1ba75a142f3f568250a544b561434a33109958f0842"} Jul 09 22:14:35.013360 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.013331 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/0.log" Jul 09 22:14:35.013483 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.013377 2575 generic.go:356] "Generic (PLEG): container finished" podID="e35b1e39-6bb4-417a-b8a6-a1560095a0c8" containerID="32edcc2559d54cf7f892ca2d6a77c915af98ba890165977c0cb6b7b6e7499a4e" exitCode=255 Jul 09 22:14:35.013483 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.013415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" event={"ID":"e35b1e39-6bb4-417a-b8a6-a1560095a0c8","Type":"ContainerDied","Data":"32edcc2559d54cf7f892ca2d6a77c915af98ba890165977c0cb6b7b6e7499a4e"} Jul 09 22:14:35.013648 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.013632 2575 scope.go:117] "RemoveContainer" containerID="32edcc2559d54cf7f892ca2d6a77c915af98ba890165977c0cb6b7b6e7499a4e" Jul 09 22:14:35.066740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.066691 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" podStartSLOduration=1.586156165 podStartE2EDuration="4.066670352s" totalImagesPullingTime="2.480514187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:35.066610688 +0000 UTC m=+243.758184741" watchObservedRunningTime="2026-07-09 22:14:35.066670352 +0000 UTC m=+243.758244407" Jul 09 22:14:35.127659 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.127599 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:35.197343 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.197312 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-clgxj"] Jul 09 22:14:35.200467 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.200447 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.203681 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.203658 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rqrfq\"" Jul 09 22:14:35.223009 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.222989 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 22:14:35.223121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.223015 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 22:14:35.249518 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.249500 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clgxj"] Jul 09 22:14:35.313501 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.313443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgwqc\" (UniqueName: \"kubernetes.io/projected/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-api-access-qgwqc\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.313595 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.313509 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-crio-socket\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.313595 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.313533 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.313675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.313640 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.313718 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.313694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-data-volume\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.349601 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.349573 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:35.414340 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414312 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-data-volume\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgwqc\" (UniqueName: \"kubernetes.io/projected/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-api-access-qgwqc\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414366 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-crio-socket\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414425 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-crio-socket\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414617 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.414617 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.414575 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:14:35.414617 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.414598 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:35.414747 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.414651 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:35.914635337 +0000 UTC m=+244.606209408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : configmap "kube-rbac-proxy" not found Jul 09 22:14:35.414747 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.414668 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:35.914660802 +0000 UTC m=+244.606234833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:35.414747 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.414687 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-data-volume\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.434434 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.434409 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgwqc\" (UniqueName: \"kubernetes.io/projected/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-api-access-qgwqc\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.572311 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.572234 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:35.827300 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.827209 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:35.919229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.919201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.919403 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:35.919273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:35.919403 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.919367 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:35.919403 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.919390 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:14:35.919554 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.919447 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:36.919432524 +0000 UTC m=+245.611006556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : configmap "kube-rbac-proxy" not found Jul 09 22:14:35.919554 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:35.919461 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:36.919455269 +0000 UTC m=+245.611029299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:36.020296 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.020267 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:14:36.020653 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.020638 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/0.log" Jul 09 22:14:36.020713 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.020676 2575 generic.go:356] "Generic (PLEG): container finished" podID="e35b1e39-6bb4-417a-b8a6-a1560095a0c8" containerID="78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4" exitCode=255 Jul 09 22:14:36.020748 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.020709 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" event={"ID":"e35b1e39-6bb4-417a-b8a6-a1560095a0c8","Type":"ContainerDied","Data":"78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4"} Jul 09 22:14:36.020788 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.020764 2575 scope.go:117] "RemoveContainer" containerID="32edcc2559d54cf7f892ca2d6a77c915af98ba890165977c0cb6b7b6e7499a4e" Jul 09 22:14:36.021116 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.021090 2575 scope.go:117] "RemoveContainer" containerID="78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4" Jul 09 22:14:36.021252 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:36.021234 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lrvcq_openshift-console-operator(e35b1e39-6bb4-417a-b8a6-a1560095a0c8)\"" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" podUID="e35b1e39-6bb4-417a-b8a6-a1560095a0c8" Jul 09 22:14:36.048479 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.048455 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:36.194516 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.194490 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:36.926511 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.926481 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:36.926915 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.926534 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:36.926915 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:36.926653 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:36.926915 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:36.926713 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:38.926699709 +0000 UTC m=+247.618273742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:36.926915 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:36.926659 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:14:36.926915 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:36.926788 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:38.926775393 +0000 UTC m=+247.618349439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : configmap "kube-rbac-proxy" not found Jul 09 22:14:36.968162 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:36.968141 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7r2hf_075e707e-55d1-476f-b14c-4875d35f240d/dns-node-resolver/0.log" Jul 09 22:14:37.025290 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.025271 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:14:37.025598 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.025575 2575 scope.go:117] "RemoveContainer" containerID="78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4" Jul 09 22:14:37.025747 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:37.025729 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lrvcq_openshift-console-operator(e35b1e39-6bb4-417a-b8a6-a1560095a0c8)\"" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" podUID="e35b1e39-6bb4-417a-b8a6-a1560095a0c8" Jul 09 22:14:37.400932 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.400905 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-s4jsp"] Jul 09 22:14:37.404678 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.404664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.408077 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.408059 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 22:14:37.408186 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.408161 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 22:14:37.408249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.408191 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 22:14:37.408996 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.408980 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-khk8j\"" Jul 09 22:14:37.409078 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.408996 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 22:14:37.426311 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.426284 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-s4jsp"] Jul 09 22:14:37.471544 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.471518 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:37.531299 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.531274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-key\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.531403 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.531303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfgq\" (UniqueName: \"kubernetes.io/projected/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-kube-api-access-8nfgq\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.531403 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.531361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-cabundle\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.632455 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.632423 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-cabundle\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.632582 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.632510 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-key\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.632618 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.632589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfgq\" (UniqueName: \"kubernetes.io/projected/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-kube-api-access-8nfgq\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.633128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.633105 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-cabundle\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.634799 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.634770 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-signing-key\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.643191 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.643171 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfgq\" (UniqueName: \"kubernetes.io/projected/b047c7fc-0dfd-4393-867e-4233a0d0fc2f-kube-api-access-8nfgq\") pod \"service-ca-758f599c5-s4jsp\" (UID: \"b047c7fc-0dfd-4393-867e-4233a0d0fc2f\") " pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.713495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.713447 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-s4jsp" Jul 09 22:14:37.838078 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:37.838024 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-s4jsp"] Jul 09 22:14:37.842521 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:37.842498 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb047c7fc_0dfd_4393_867e_4233a0d0fc2f.slice/crio-f2c0591446329fcc64b6c5f9c6297087e8fbdf14b7258ffed3ecaf8cc28f2f0e WatchSource:0}: Error finding container f2c0591446329fcc64b6c5f9c6297087e8fbdf14b7258ffed3ecaf8cc28f2f0e: Status 404 returned error can't find the container with id f2c0591446329fcc64b6c5f9c6297087e8fbdf14b7258ffed3ecaf8cc28f2f0e Jul 09 22:14:38.029620 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.029542 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-s4jsp" event={"ID":"b047c7fc-0dfd-4393-867e-4233a0d0fc2f","Type":"ContainerStarted","Data":"f2c0591446329fcc64b6c5f9c6297087e8fbdf14b7258ffed3ecaf8cc28f2f0e"} Jul 09 22:14:38.370699 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.370621 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sh9k4_5a3da10f-2ff6-4343-9843-cd58a6fd7339/node-ca/0.log" Jul 09 22:14:38.742785 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.742752 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:38.743003 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.742976 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:14:38.743003 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.743001 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776765f57c-9mb8c: secret "image-registry-tls" not found Jul 09 22:14:38.743152 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.743067 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls podName:a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:46.74304629 +0000 UTC m=+255.434620324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls") pod "image-registry-776765f57c-9mb8c" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0") : secret "image-registry-tls" not found Jul 09 22:14:38.777907 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.777873 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d479r_b45a8325-8a9b-41f0-ba02-0b31ddd5aea1/serve-healthcheck-canary/0.log" Jul 09 22:14:38.843673 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.843644 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:38.843801 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.843756 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:38.843839 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.843805 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:14:46.843793264 +0000 UTC m=+255.535367294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:38.929886 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.929853 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:38.944939 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.944909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:38.945079 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:38.944968 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:38.945079 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.945039 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:14:38.945079 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.945053 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:38.945233 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.945093 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:42.945078152 +0000 UTC m=+251.636652185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : configmap "kube-rbac-proxy" not found Jul 09 22:14:38.945233 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:38.945113 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:42.945103798 +0000 UTC m=+251.636677834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:39.329047 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:39.329015 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:40.037199 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:40.037170 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-s4jsp" event={"ID":"b047c7fc-0dfd-4393-867e-4233a0d0fc2f","Type":"ContainerStarted","Data":"3babde49d60eb2329281d7dde7b1209ef5385fa2484a96ae8fee9c8aad5956aa"} Jul 09 22:14:41.413638 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:41.413604 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:41.414090 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:41.413736 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:41.414209 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:41.414189 2575 scope.go:117] "RemoveContainer" containerID="78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4" Jul 09 22:14:41.414398 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:41.414376 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lrvcq_openshift-console-operator(e35b1e39-6bb4-417a-b8a6-a1560095a0c8)\"" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" podUID="e35b1e39-6bb4-417a-b8a6-a1560095a0c8" Jul 09 22:14:41.597953 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:41.597911 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:42.083104 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:42.083068 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:42.363370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:42.363286 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:42.464910 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:42.464866 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:42.977841 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:42.977811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:42.978040 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:42.977867 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:42.978040 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:42.977971 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:14:42.978040 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:42.978012 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:42.978139 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:42.978055 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:50.97803758 +0000 UTC m=+259.669611611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : configmap "kube-rbac-proxy" not found Jul 09 22:14:42.978139 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:42.978070 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:14:50.97806381 +0000 UTC m=+259.669637840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:43.085047 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:43.085018 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:43.677013 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:43.676979 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:43.870003 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:43.869969 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:44.078302 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:44.078230 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:44.087536 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:44.087515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:14:44.087676 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:44.087660 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:14:44.087737 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:44.087727 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs podName:4361956b-0edf-45fb-90e6-ab52b1b3286b nodeName:}" failed. No retries permitted until 2026-07-09 22:16:46.087711979 +0000 UTC m=+374.779286009 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs") pod "network-metrics-daemon-pwcd7" (UID: "4361956b-0edf-45fb-90e6-ab52b1b3286b") : secret "metrics-daemon-secret" not found Jul 09 22:14:44.474262 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:44.474233 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:44.564951 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:44.564924 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:45.283147 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:45.283113 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:45.387326 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:45.387297 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:45.496988 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:45.496957 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:45.670135 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:45.670109 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:46.067316 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.067231 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:46.483144 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.483089 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:46.810433 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.810338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:46.812909 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.812866 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"image-registry-776765f57c-9mb8c\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:46.862460 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.862431 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jn276\"" Jul 09 22:14:46.868990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.868972 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:46.911232 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.911181 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:14:46.911432 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:46.911361 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:46.911492 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:46.911443 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls podName:84465f8d-067f-407e-9efb-aeec5b9d16c0 nodeName:}" failed. No retries permitted until 2026-07-09 22:15:02.911426343 +0000 UTC m=+271.603000387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-twgjb" (UID: "84465f8d-067f-407e-9efb-aeec5b9d16c0") : secret "cluster-monitoring-operator-tls" not found Jul 09 22:14:46.995844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.995755 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-s4jsp" podStartSLOduration=7.91600953 podStartE2EDuration="9.99573796s" totalImagesPullingTime="2.07972843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:40.06421798 +0000 UTC m=+248.755792055" watchObservedRunningTime="2026-07-09 22:14:46.99573796 +0000 UTC m=+255.687312021" Jul 09 22:14:46.996741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:46.996723 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:14:47.001187 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:14:47.001155 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ac9aaa_bb4c_427e_a4a6_8c700c86e6c0.slice/crio-03f8d791b89d888858850feb549bc473e78b1d05a98d0ae27f59f15a5a79ea92 WatchSource:0}: Error finding container 03f8d791b89d888858850feb549bc473e78b1d05a98d0ae27f59f15a5a79ea92: Status 404 returned error can't find the container with id 03f8d791b89d888858850feb549bc473e78b1d05a98d0ae27f59f15a5a79ea92 Jul 09 22:14:47.059571 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:47.059539 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" event={"ID":"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0","Type":"ContainerStarted","Data":"03f8d791b89d888858850feb549bc473e78b1d05a98d0ae27f59f15a5a79ea92"} Jul 09 22:14:47.256246 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:47.256218 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:47.674982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:47.674956 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:47.855752 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:47.855715 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:47.994738 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:47.994663 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:48.063789 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.063762 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" event={"ID":"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0","Type":"ContainerStarted","Data":"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03"} Jul 09 22:14:48.063965 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.063884 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:14:48.087607 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.087556 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" podStartSLOduration=18.087538856 podStartE2EDuration="18.087538856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:48.086818309 +0000 UTC m=+256.778392362" watchObservedRunningTime="2026-07-09 22:14:48.087538856 +0000 UTC m=+256.779112911" Jul 09 22:14:48.090942 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.090923 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:48.255251 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.255180 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:48.432243 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:48.432187 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:49.254550 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:49.254520 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:50.120748 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:50.120715 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:50.201239 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:50.201208 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:50.454510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:50.454484 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:50.662703 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:50.662672 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:50.755132 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:50.755049 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:51.045599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.045526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:51.045599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.045577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:51.045823 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:51.045695 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:14:51.045823 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:14:51.045770 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls podName:bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec nodeName:}" failed. No retries permitted until 2026-07-09 22:15:07.045748842 +0000 UTC m=+275.737322891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clgxj" (UID: "bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec") : secret "insights-runtime-extractor-tls" not found Jul 09 22:14:51.046103 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.046084 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:14:51.087475 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.087448 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:51.463641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.463613 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:51.886681 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.886603 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:51.995305 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:51.995273 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:52.096580 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:52.096550 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:52.280843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:52.280818 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:54.186029 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:54.185959 2575 scope.go:117] "RemoveContainer" containerID="78f20c5d0705495ab6746d772ebfe8e50b49d17a8d427c4f5ecbaf7fdd68f2a4" Jul 09 22:14:55.061727 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.061687 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:55.086685 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.086658 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:14:55.086811 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.086725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" event={"ID":"e35b1e39-6bb4-417a-b8a6-a1560095a0c8","Type":"ContainerStarted","Data":"ba9743d1d24e0e10e707af1e5e13c0a498194567b978cdc90d59b7040302d07a"} Jul 09 22:14:55.087024 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.087003 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:55.091503 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.091483 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" Jul 09 22:14:55.158331 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.158293 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-lrvcq" podStartSLOduration=21.674644916 podStartE2EDuration="24.15828179s" totalImagesPullingTime="2.483636874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:55.157100897 +0000 UTC m=+263.848674949" watchObservedRunningTime="2026-07-09 22:14:55.15828179 +0000 UTC m=+263.849855842" Jul 09 22:14:55.242447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.242423 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:55.363610 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.363530 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:55.464587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.464558 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:55.744049 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:55.744008 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:56.011753 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:56.011678 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:56.615343 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:56.615306 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:56.727122 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:56.727094 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:56.830406 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:56.830378 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:56.947611 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:56.947580 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:57.381155 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:57.381083 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:14:57.463514 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:14:57.463487 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:02.106221 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.106184 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:15:02.110846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.110817 2575 patch_prober.go:28] interesting pod/image-registry-776765f57c-9mb8c container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 22:15:02.111005 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.110872 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:15:02.247458 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.247427 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-779764dffc-8l7hg"] Jul 09 22:15:02.251629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.251613 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.298365 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.298330 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779764dffc-8l7hg"] Jul 09 22:15:02.386877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.386797 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:02.434177 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-bound-sa-token\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434177 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434176 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4lzs\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-kube-api-access-x4lzs\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434205 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-image-registry-private-configuration\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-certificates\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434265 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-trusted-ca\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-installation-pull-secrets\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434306 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-ca-trust-extracted\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.434352 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.434351 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-tls\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535668 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535634 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-bound-sa-token\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535668 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4lzs\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-kube-api-access-x4lzs\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-image-registry-private-configuration\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-certificates\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-trusted-ca\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-installation-pull-secrets\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535819 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-ca-trust-extracted\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.535877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.535860 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-tls\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.536358 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.536324 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-ca-trust-extracted\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.536776 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.536750 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-certificates\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.536974 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.536957 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-trusted-ca\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.538313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.538291 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-image-registry-private-configuration\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.538525 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.538504 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-registry-tls\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.538571 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.538535 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-installation-pull-secrets\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.548795 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.548772 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4lzs\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-kube-api-access-x4lzs\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.549669 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.549650 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/358db4b1-0adf-45db-a5ec-fc9e2f8aa446-bound-sa-token\") pod \"image-registry-779764dffc-8l7hg\" (UID: \"358db4b1-0adf-45db-a5ec-fc9e2f8aa446\") " pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.559952 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.559928 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:02.560043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.560030 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:02.706769 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.706742 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779764dffc-8l7hg"] Jul 09 22:15:02.709848 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:02.709823 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358db4b1_0adf_45db_a5ec_fc9e2f8aa446.slice/crio-064e9e9e9421d54ccdb16fd5e33c3d6196889f365ab0af2f757be73849bf9f1c WatchSource:0}: Error finding container 064e9e9e9421d54ccdb16fd5e33c3d6196889f365ab0af2f757be73849bf9f1c: Status 404 returned error can't find the container with id 064e9e9e9421d54ccdb16fd5e33c3d6196889f365ab0af2f757be73849bf9f1c Jul 09 22:15:02.938000 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.937969 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:15:02.940228 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:02.940208 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84465f8d-067f-407e-9efb-aeec5b9d16c0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-twgjb\" (UID: \"84465f8d-067f-407e-9efb-aeec5b9d16c0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:15:03.112909 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.112852 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" event={"ID":"358db4b1-0adf-45db-a5ec-fc9e2f8aa446","Type":"ContainerStarted","Data":"37c41e6628853689d1244dc90c5a505048e8d3bbc63c08230c0ac5831d97c068"} Jul 09 22:15:03.113258 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.112961 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" event={"ID":"358db4b1-0adf-45db-a5ec-fc9e2f8aa446","Type":"ContainerStarted","Data":"064e9e9e9421d54ccdb16fd5e33c3d6196889f365ab0af2f757be73849bf9f1c"} Jul 09 22:15:03.113258 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.112989 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:03.146517 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.146471 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" podStartSLOduration=1.1464554869999999 podStartE2EDuration="1.146455487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:03.143867202 +0000 UTC m=+271.835441268" watchObservedRunningTime="2026-07-09 22:15:03.146455487 +0000 UTC m=+271.838029540" Jul 09 22:15:03.220873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.220806 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d8sq5\"" Jul 09 22:15:03.227327 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.227300 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" Jul 09 22:15:03.366798 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:03.366763 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb"] Jul 09 22:15:03.369841 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:03.369813 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84465f8d_067f_407e_9efb_aeec5b9d16c0.slice/crio-06ef10935281e4241a52558cbfca69539bfc8de5dd721cf67d158e316beed866 WatchSource:0}: Error finding container 06ef10935281e4241a52558cbfca69539bfc8de5dd721cf67d158e316beed866: Status 404 returned error can't find the container with id 06ef10935281e4241a52558cbfca69539bfc8de5dd721cf67d158e316beed866 Jul 09 22:15:04.116859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:04.116825 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" event={"ID":"84465f8d-067f-407e-9efb-aeec5b9d16c0","Type":"ContainerStarted","Data":"06ef10935281e4241a52558cbfca69539bfc8de5dd721cf67d158e316beed866"} Jul 09 22:15:05.815368 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:05.815333 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:05.940856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:05.940823 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:06.126123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:06.126038 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" event={"ID":"84465f8d-067f-407e-9efb-aeec5b9d16c0","Type":"ContainerStarted","Data":"c5cf996db86c5fd1decd2d171e6ef1dd7716ff5989e622dd8d916e423b428533"} Jul 09 22:15:06.160727 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:06.160695 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:06.161914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:06.161846 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" podStartSLOduration=33.203366006 podStartE2EDuration="35.161832281s" totalImagesPullingTime="1.958466275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:06.161304903 +0000 UTC m=+274.852878959" watchObservedRunningTime="2026-07-09 22:15:06.161832281 +0000 UTC m=+274.853406333" Jul 09 22:15:06.545698 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:06.545666 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:07.067552 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:07.067512 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:15:07.069722 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:07.069701 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clgxj\" (UID: \"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec\") " pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:15:07.312861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:07.312828 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clgxj" Jul 09 22:15:07.456940 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:07.456883 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clgxj"] Jul 09 22:15:07.460401 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:07.460373 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdceb3e1_b99f_4a9e_b8cc_6db512d9f7ec.slice/crio-f37a01c46cefb3eaefcbf3b01008bcfe4af5858df592f2f7f02cfc81634c62c3 WatchSource:0}: Error finding container f37a01c46cefb3eaefcbf3b01008bcfe4af5858df592f2f7f02cfc81634c62c3: Status 404 returned error can't find the container with id f37a01c46cefb3eaefcbf3b01008bcfe4af5858df592f2f7f02cfc81634c62c3 Jul 09 22:15:08.132969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:08.132935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clgxj" event={"ID":"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec","Type":"ContainerStarted","Data":"f37a01c46cefb3eaefcbf3b01008bcfe4af5858df592f2f7f02cfc81634c62c3"} Jul 09 22:15:09.005720 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:09.005682 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:09.105102 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:09.105066 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:09.452123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:09.452086 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:09.945883 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:09.945846 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:12.105244 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:12.105144 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:12.110709 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:12.110690 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:15:12.148569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:12.148538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clgxj" event={"ID":"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec","Type":"ContainerStarted","Data":"1c2953205427401e06950cfba433185bd76a4edf1ee51e952a54d3830423d32e"} Jul 09 22:15:12.203353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:12.203330 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:13.153755 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:13.153712 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clgxj" event={"ID":"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec","Type":"ContainerStarted","Data":"b91af6c9b193ef0890fe610ac215b6c60720e3565dbc1ef109e2123055329c22"} Jul 09 22:15:13.777304 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:13.777259 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b4w94" podUID="b8f0f8e8-a18a-450e-81af-c9385ea3c9a7" Jul 09 22:15:14.157105 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:14.157079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:15.162253 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.162214 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clgxj" event={"ID":"bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec","Type":"ContainerStarted","Data":"b92501df307b110d0db7b028e4961790c3b126937bb38a5f83ca1a5f072801ee"} Jul 09 22:15:15.187834 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.187791 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.195112 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.195074 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-clgxj" podStartSLOduration=32.999602215 podStartE2EDuration="40.195060454s" totalImagesPullingTime="7.195458239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:14:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:15.193813645 +0000 UTC m=+283.885387699" watchObservedRunningTime="2026-07-09 22:15:15.195060454 +0000 UTC m=+283.886634507" Jul 09 22:15:15.284827 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.284785 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.382912 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.382866 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.592673 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.592602 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.695148 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.695116 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.785986 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.785952 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:15.894081 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:15.893998 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:16.615740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:16.615708 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:16.716092 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:16.716064 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:17.148738 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:17.148696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:17.151064 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:17.151043 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8f0f8e8-a18a-450e-81af-c9385ea3c9a7-metrics-tls\") pod \"dns-default-b4w94\" (UID: \"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7\") " pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:17.160909 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:17.160875 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r6x6z\"" Jul 09 22:15:17.168147 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:17.168127 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:17.292742 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:17.292710 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b4w94"] Jul 09 22:15:17.295384 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:17.295358 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f0f8e8_a18a_450e_81af_c9385ea3c9a7.slice/crio-bcd78a11beb144eb3299827f1e68dcbc211828e15a7b1ebcf3c2454b7fe5bf93 WatchSource:0}: Error finding container bcd78a11beb144eb3299827f1e68dcbc211828e15a7b1ebcf3c2454b7fe5bf93: Status 404 returned error can't find the container with id bcd78a11beb144eb3299827f1e68dcbc211828e15a7b1ebcf3c2454b7fe5bf93 Jul 09 22:15:18.171546 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:18.171508 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b4w94" event={"ID":"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7","Type":"ContainerStarted","Data":"bcd78a11beb144eb3299827f1e68dcbc211828e15a7b1ebcf3c2454b7fe5bf93"} Jul 09 22:15:20.180032 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.179991 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b4w94" event={"ID":"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7","Type":"ContainerStarted","Data":"fca4066724901f7dcf17498ff6350a4add9ae96086a495ec58c1956496b41dec"} Jul 09 22:15:20.180453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.180052 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b4w94" event={"ID":"b8f0f8e8-a18a-450e-81af-c9385ea3c9a7","Type":"ContainerStarted","Data":"a1467ababa1a81a85ae17faaf8b0d8ba1630941cb8334f0246331941a2d16fae"} Jul 09 22:15:20.180453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.180092 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:20.200476 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.200432 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b4w94" podStartSLOduration=250.934566215 podStartE2EDuration="4m13.20041785s" totalImagesPullingTime="2.265851635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:20.198996144 +0000 UTC m=+288.890570197" watchObservedRunningTime="2026-07-09 22:15:20.20041785 +0000 UTC m=+288.891991902" Jul 09 22:15:20.637063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.636987 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:20.736322 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:20.736288 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:21.256134 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:21.256087 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:21.352158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:21.352111 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:23.381667 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:23.381587 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:23.453745 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:23.453712 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:24.121251 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:24.121225 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-779764dffc-8l7hg" Jul 09 22:15:24.312051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:24.312021 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:24.380950 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:24.380864 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:24.451917 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:24.451878 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:25.180226 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.180191 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:25.202099 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.202080 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:15:25.202216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.202113 2575 generic.go:356] "Generic (PLEG): container finished" podID="84465f8d-067f-407e-9efb-aeec5b9d16c0" containerID="c5cf996db86c5fd1decd2d171e6ef1dd7716ff5989e622dd8d916e423b428533" exitCode=2 Jul 09 22:15:25.202216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.202170 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" event={"ID":"84465f8d-067f-407e-9efb-aeec5b9d16c0","Type":"ContainerDied","Data":"c5cf996db86c5fd1decd2d171e6ef1dd7716ff5989e622dd8d916e423b428533"} Jul 09 22:15:25.202457 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.202445 2575 scope.go:117] "RemoveContainer" containerID="c5cf996db86c5fd1decd2d171e6ef1dd7716ff5989e622dd8d916e423b428533" Jul 09 22:15:25.405291 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:25.405256 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:26.207396 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:26.207372 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:15:26.207744 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:26.207440 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-twgjb" event={"ID":"84465f8d-067f-407e-9efb-aeec5b9d16c0","Type":"ContainerStarted","Data":"4b233b786f993926dfe652d47b2e24714523c0dc4681cb1ba4e6a76fada55e0f"} Jul 09 22:15:26.590151 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:26.590072 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:26.686163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:26.686131 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:26.786434 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:26.786405 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:27.125822 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.125783 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerName="registry" containerID="cri-o://262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03" gracePeriod=30 Jul 09 22:15:27.187057 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:27.187021 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pwcd7" podUID="4361956b-0edf-45fb-90e6-ab52b1b3286b" Jul 09 22:15:27.358530 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.358508 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:15:27.435148 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435121 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435288 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435159 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435288 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435216 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgmxm\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435288 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435244 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435474 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435287 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435474 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435321 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435474 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435349 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435474 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435394 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets\") pod \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\" (UID: \"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0\") " Jul 09 22:15:27.435662 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435634 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:15:27.435717 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.435683 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:15:27.437617 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.437595 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:15:27.437772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.437756 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:15:27.437842 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.437776 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:15:27.437842 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.437797 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:15:27.437953 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.437868 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm" (OuterVolumeSpecName: "kube-api-access-tgmxm") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "kube-api-access-tgmxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:15:27.444162 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.444141 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" (UID: "a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:15:27.535912 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535863 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgmxm\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-kube-api-access-tgmxm\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.535912 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535886 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-certificates\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.535912 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535912 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-ca-trust-extracted\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.536080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535921 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-image-registry-private-configuration\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.536080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535931 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-registry-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.536080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535940 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-installation-pull-secrets\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.536080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535949 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-bound-sa-token\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:27.536080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:27.535957 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0-trusted-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:15:28.215760 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.215724 2575 generic.go:356] "Generic (PLEG): container finished" podID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerID="262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03" exitCode=0 Jul 09 22:15:28.215976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.215777 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" Jul 09 22:15:28.215976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.215779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" event={"ID":"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0","Type":"ContainerDied","Data":"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03"} Jul 09 22:15:28.215976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.215915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776765f57c-9mb8c" event={"ID":"a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0","Type":"ContainerDied","Data":"03f8d791b89d888858850feb549bc473e78b1d05a98d0ae27f59f15a5a79ea92"} Jul 09 22:15:28.215976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.215944 2575 scope.go:117] "RemoveContainer" containerID="262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03" Jul 09 22:15:28.225530 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.225507 2575 scope.go:117] "RemoveContainer" containerID="262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03" Jul 09 22:15:28.225812 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:28.225789 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03\": container with ID starting with 262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03 not found: ID does not exist" containerID="262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03" Jul 09 22:15:28.225872 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.225822 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03"} err="failed to get container status \"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03\": rpc error: code = NotFound desc = could not find container \"262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03\": container with ID starting with 262517409d3cfc208207601e1cf39ed02608bdefb802f7767de4499c9fd96f03 not found: ID does not exist" Jul 09 22:15:28.241628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.241609 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:15:28.246073 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.246054 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-776765f57c-9mb8c"] Jul 09 22:15:28.915783 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.915748 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:28.926592 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.926560 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd"] Jul 09 22:15:28.926954 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.926923 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerName="registry" Jul 09 22:15:28.926954 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.926954 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerName="registry" Jul 09 22:15:28.927128 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.927060 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" containerName="registry" Jul 09 22:15:28.932326 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.932305 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5sw4w"] Jul 09 22:15:28.932427 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.932411 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:28.935440 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.935420 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:28.935545 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.935526 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gjp2w\"" Jul 09 22:15:28.936772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.936749 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 22:15:28.936861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.936800 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 22:15:28.936861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.936802 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 22:15:28.938238 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.938219 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tq6cx\"" Jul 09 22:15:28.938647 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.938626 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 22:15:28.938860 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.938841 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 22:15:28.940155 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.940135 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 22:15:28.943141 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.943121 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd"] Jul 09 22:15:28.949959 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.949940 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz"] Jul 09 22:15:28.953305 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.953288 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:28.957202 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.957182 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 22:15:28.957304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.957234 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nrv5r\"" Jul 09 22:15:28.957536 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.957516 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 22:15:28.957989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.957963 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 22:15:28.969028 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:28.969009 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz"] Jul 09 22:15:29.006229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.006208 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:29.046109 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-sys\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-root\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-tls\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e588d5f-07ca-4d38-aefc-7c225eceb236-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.046215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.046383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-textfile\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.046383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-metrics-client-ca\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046369 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.046534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046417 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tglxp\" (UniqueName: \"kubernetes.io/projected/7e588d5f-07ca-4d38-aefc-7c225eceb236-kube-api-access-tglxp\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.046534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txrzj\" (UniqueName: \"kubernetes.io/projected/1e0eb537-921f-4d36-9c87-ca16c0f92207-kube-api-access-txrzj\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-wtmp\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046671 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046546 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.046671 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4npn\" (UniqueName: \"kubernetes.io/projected/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-api-access-l4npn\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.046671 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046613 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-accelerators-collector-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.046780 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.046780 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.046780 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.046752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/29fb3c25-e31d-43be-a93f-f6692ec798f0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.127310 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.127275 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:29.147515 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147485 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/29fb3c25-e31d-43be-a93f-f6692ec798f0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.147639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-sys\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-root\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-tls\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147618 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e588d5f-07ca-4d38-aefc-7c225eceb236-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147643 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-sys\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147689 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-textfile\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147724 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-metrics-client-ca\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147781 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.147850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tglxp\" (UniqueName: \"kubernetes.io/projected/7e588d5f-07ca-4d38-aefc-7c225eceb236-kube-api-access-tglxp\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.148207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147857 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147887 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txrzj\" (UniqueName: \"kubernetes.io/projected/1e0eb537-921f-4d36-9c87-ca16c0f92207-kube-api-access-txrzj\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147946 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-wtmp\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.148207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147999 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/29fb3c25-e31d-43be-a93f-f6692ec798f0-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.148495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4npn\" (UniqueName: \"kubernetes.io/projected/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-api-access-l4npn\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.148495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148263 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-accelerators-collector-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148309 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.148495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.148495 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148393 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e588d5f-07ca-4d38-aefc-7c225eceb236-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.148756 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148506 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-wtmp\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148756 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.147647 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1e0eb537-921f-4d36-9c87-ca16c0f92207-root\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.148854 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.148774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-textfile\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.149024 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.149001 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-accelerators-collector-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.149180 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.149156 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.149315 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.149292 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0eb537-921f-4d36-9c87-ca16c0f92207-metrics-client-ca\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.149498 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.149479 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.150566 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.150521 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.151271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.151229 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.151271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.151242 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-tls\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.151419 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.151389 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.151728 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.151708 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e588d5f-07ca-4d38-aefc-7c225eceb236-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.152069 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.152052 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0eb537-921f-4d36-9c87-ca16c0f92207-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.157540 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.157521 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txrzj\" (UniqueName: \"kubernetes.io/projected/1e0eb537-921f-4d36-9c87-ca16c0f92207-kube-api-access-txrzj\") pod \"node-exporter-5sw4w\" (UID: \"1e0eb537-921f-4d36-9c87-ca16c0f92207\") " pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.159078 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.159062 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4npn\" (UniqueName: \"kubernetes.io/projected/29fb3c25-e31d-43be-a93f-f6692ec798f0-kube-api-access-l4npn\") pod \"kube-state-metrics-7c9dbb8b7d-xj4rz\" (UID: \"29fb3c25-e31d-43be-a93f-f6692ec798f0\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.163031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.163012 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tglxp\" (UniqueName: \"kubernetes.io/projected/7e588d5f-07ca-4d38-aefc-7c225eceb236-kube-api-access-tglxp\") pod \"openshift-state-metrics-54cf96dfcf-9djpd\" (UID: \"7e588d5f-07ca-4d38-aefc-7c225eceb236\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.190649 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.190588 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0" path="/var/lib/kubelet/pods/a5ac9aaa-bb4c-427e-a4a6-8c700c86e6c0/volumes" Jul 09 22:15:29.225526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.225503 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:29.244969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.244747 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" Jul 09 22:15:29.249708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.249262 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5sw4w" Jul 09 22:15:29.262502 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.262473 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" Jul 09 22:15:29.268747 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:29.268701 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0eb537_921f_4d36_9c87_ca16c0f92207.slice/crio-650bdee3cd86692cac4bb52d80e3043dcffd78f5be8873f14ec6ff24ba03ad3e WatchSource:0}: Error finding container 650bdee3cd86692cac4bb52d80e3043dcffd78f5be8873f14ec6ff24ba03ad3e: Status 404 returned error can't find the container with id 650bdee3cd86692cac4bb52d80e3043dcffd78f5be8873f14ec6ff24ba03ad3e Jul 09 22:15:29.327598 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.327560 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:29.394820 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.394795 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd"] Jul 09 22:15:29.397401 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:29.397355 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e588d5f_07ca_4d38_aefc_7c225eceb236.slice/crio-d7824457030ce108cd292bde1f7d7593563093427941536714e0eddd978fb56e WatchSource:0}: Error finding container d7824457030ce108cd292bde1f7d7593563093427941536714e0eddd978fb56e: Status 404 returned error can't find the container with id d7824457030ce108cd292bde1f7d7593563093427941536714e0eddd978fb56e Jul 09 22:15:29.397588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.397566 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:29.416258 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.416232 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz"] Jul 09 22:15:29.430041 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:29.430017 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fb3c25_e31d_43be_a93f_f6692ec798f0.slice/crio-582f69b878e3a877e8160edc5ba40333d0085adc22533215dff3b66d321f344f WatchSource:0}: Error finding container 582f69b878e3a877e8160edc5ba40333d0085adc22533215dff3b66d321f344f: Status 404 returned error can't find the container with id 582f69b878e3a877e8160edc5ba40333d0085adc22533215dff3b66d321f344f Jul 09 22:15:29.905431 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:29.905400 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:30.007547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.007456 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:30.012588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.012557 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 22:15:30.017072 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.017048 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.019867 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.019847 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 22:15:30.020208 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020193 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 22:15:30.020316 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020299 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020255 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020599 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020601 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020650 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020609 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020601 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 22:15:30.020750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.020709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ztbnv\"" Jul 09 22:15:30.031181 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.031159 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 22:15:30.156453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156444 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-config-out\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156462 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xx7q\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-kube-api-access-2xx7q\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156478 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156494 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-web-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.156707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.157008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.157008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156805 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.157008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156840 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.157008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.156873 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.186371 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.186349 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b4w94" Jul 09 22:15:30.240922 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.238915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" event={"ID":"29fb3c25-e31d-43be-a93f-f6692ec798f0","Type":"ContainerStarted","Data":"582f69b878e3a877e8160edc5ba40333d0085adc22533215dff3b66d321f344f"} Jul 09 22:15:30.245944 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.245885 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5sw4w" event={"ID":"1e0eb537-921f-4d36-9c87-ca16c0f92207","Type":"ContainerStarted","Data":"650bdee3cd86692cac4bb52d80e3043dcffd78f5be8873f14ec6ff24ba03ad3e"} Jul 09 22:15:30.248453 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.248425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" event={"ID":"7e588d5f-07ca-4d38-aefc-7c225eceb236","Type":"ContainerStarted","Data":"81277fa3a2b9953fb19c7c01ba55f3c9b97e84ebed27c73e13e71e930c0a9517"} Jul 09 22:15:30.248586 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.248477 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" event={"ID":"7e588d5f-07ca-4d38-aefc-7c225eceb236","Type":"ContainerStarted","Data":"0d00b1e2bf5c587437ba846fb7b7ec8f02d217049f3f420a3fe9be05e3c44700"} Jul 09 22:15:30.248586 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.248505 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" event={"ID":"7e588d5f-07ca-4d38-aefc-7c225eceb236","Type":"ContainerStarted","Data":"d7824457030ce108cd292bde1f7d7593563093427941536714e0eddd978fb56e"} Jul 09 22:15:30.258074 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258120 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258238 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258281 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-config-out\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xx7q\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-kube-api-access-2xx7q\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258357 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258395 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258427 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-web-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.258587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.258450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.259800 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:30.259251 2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 22:15:30.259800 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:30.259310 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls podName:35691cf3-c17e-429d-9f0f-6347619e03e2 nodeName:}" failed. No retries permitted until 2026-07-09 22:15:30.759292013 +0000 UTC m=+299.450866051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "35691cf3-c17e-429d-9f0f-6347619e03e2") : secret "alertmanager-main-tls" not found Jul 09 22:15:30.259800 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:15:30.259575 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle podName:35691cf3-c17e-429d-9f0f-6347619e03e2 nodeName:}" failed. No retries permitted until 2026-07-09 22:15:30.759554413 +0000 UTC m=+299.451128458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "35691cf3-c17e-429d-9f0f-6347619e03e2") : configmap references non-existent config key: ca-bundle.crt Jul 09 22:15:30.260846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.260824 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.261336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.261315 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.264195 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.264131 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.264695 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.264501 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.264695 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.264645 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.265336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.264987 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-web-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.265336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.265299 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.266427 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.266401 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35691cf3-c17e-429d-9f0f-6347619e03e2-config-out\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.267303 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.267237 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.273180 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.273157 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-config-volume\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.283366 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.283345 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xx7q\" (UniqueName: \"kubernetes.io/projected/35691cf3-c17e-429d-9f0f-6347619e03e2-kube-api-access-2xx7q\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.450165 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.450057 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:30.763224 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.763145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.763224 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.763198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.764170 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.764146 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35691cf3-c17e-429d-9f0f-6347619e03e2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.765867 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.765847 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35691cf3-c17e-429d-9f0f-6347619e03e2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35691cf3-c17e-429d-9f0f-6347619e03e2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:30.930055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:30.930022 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 22:15:31.199216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.198772 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 22:15:31.255223 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.255190 2575 generic.go:356] "Generic (PLEG): container finished" podID="1e0eb537-921f-4d36-9c87-ca16c0f92207" containerID="4952633ebf0db49c1649e26484ebcff920681273051d22b51c8beede9c82903c" exitCode=0 Jul 09 22:15:31.255372 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.255277 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5sw4w" event={"ID":"1e0eb537-921f-4d36-9c87-ca16c0f92207","Type":"ContainerDied","Data":"4952633ebf0db49c1649e26484ebcff920681273051d22b51c8beede9c82903c"} Jul 09 22:15:31.258918 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.258867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" event={"ID":"7e588d5f-07ca-4d38-aefc-7c225eceb236","Type":"ContainerStarted","Data":"48dc180e34e479a58b5cc658bc231910a0a276d16fb2e259f0fe010151cdc251"} Jul 09 22:15:31.261269 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.261206 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" event={"ID":"29fb3c25-e31d-43be-a93f-f6692ec798f0","Type":"ContainerStarted","Data":"89fecf7ea9952f6e50d52fd2fb23061ee46022adcfbd2e0534116169111e5c7f"} Jul 09 22:15:31.261269 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.261259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" event={"ID":"29fb3c25-e31d-43be-a93f-f6692ec798f0","Type":"ContainerStarted","Data":"30399a5bd25f4010165c2b4d902c6f1712638b07bb29a99c27ef9b0f77239551"} Jul 09 22:15:31.262463 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.262398 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"918f226529f6fe5047a362b1b19219ee0c2006e6fac3769b7c3867e38373c192"} Jul 09 22:15:31.316326 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.316275 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9djpd" podStartSLOduration=1.73466341 podStartE2EDuration="3.316254089s" totalImagesPullingTime="1.581590679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:31.315418642 +0000 UTC m=+300.006992697" watchObservedRunningTime="2026-07-09 22:15:31.316254089 +0000 UTC m=+300.007828145" Jul 09 22:15:31.863574 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.863538 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:31.936931 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:31.936877 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:32.215932 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.215878 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:32.268697 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.268646 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5sw4w" event={"ID":"1e0eb537-921f-4d36-9c87-ca16c0f92207","Type":"ContainerStarted","Data":"2c2e57e9b64171a59643b1e8af0a192d4d7238d8cb8610450050e09af2a09bd9"} Jul 09 22:15:32.268865 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.268739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5sw4w" event={"ID":"1e0eb537-921f-4d36-9c87-ca16c0f92207","Type":"ContainerStarted","Data":"445541f8222d0b9daca8dad50d28f0c282d3d30b4e9fa3f05a66971c338b35d6"} Jul 09 22:15:32.271195 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.271155 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" event={"ID":"29fb3c25-e31d-43be-a93f-f6692ec798f0","Type":"ContainerStarted","Data":"717e6428eeb1494bc9ffc54dba2408286e42bd0227f9da3b2c346943cdbd0255"} Jul 09 22:15:32.298986 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.298931 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5sw4w" podStartSLOduration=3.083111896 podStartE2EDuration="4.298911004s" totalImagesPullingTime="1.138044492s" totalInitContainerRuntime="77.754616ms" isStatefulPod=true podCreationTimestamp="2026-07-09 22:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:32.296211623 +0000 UTC m=+300.987785680" watchObservedRunningTime="2026-07-09 22:15:32.298911004 +0000 UTC m=+300.990485055" Jul 09 22:15:32.301089 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.301060 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:32.316703 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.316677 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:15:32.317989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.317970 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:15:32.318147 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.318130 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:15:32.319367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.319350 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:15:32.322773 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.322722 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-xj4rz" podStartSLOduration=2.703504227 podStartE2EDuration="4.322708777s" totalImagesPullingTime="1.61920455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:32.322448414 +0000 UTC m=+301.014022470" watchObservedRunningTime="2026-07-09 22:15:32.322708777 +0000 UTC m=+301.014282830" Jul 09 22:15:32.323444 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.323426 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:15:32.324756 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.324740 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:15:32.329604 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:32.329586 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 22:15:33.249602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.249570 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d4d665b8-ds9hd"] Jul 09 22:15:33.252656 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.252641 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.255631 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.255600 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 22:15:33.255760 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.255613 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 22:15:33.257310 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.257294 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 22:15:33.257310 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.257299 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 22:15:33.257507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.257350 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eee9ui2v520gp\"" Jul 09 22:15:33.257507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.257431 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mbhzd\"" Jul 09 22:15:33.262302 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.262282 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d4d665b8-ds9hd"] Jul 09 22:15:33.276168 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.276143 2575 generic.go:356] "Generic (PLEG): container finished" podID="35691cf3-c17e-429d-9f0f-6347619e03e2" containerID="ed4f2f3b891b6d44f8f461287ad25fa013920f6245b58fc36ab7b7f0a8c2adb7" exitCode=0 Jul 09 22:15:33.276260 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.276228 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerDied","Data":"ed4f2f3b891b6d44f8f461287ad25fa013920f6245b58fc36ab7b7f0a8c2adb7"} Jul 09 22:15:33.387197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387151 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-client-certs\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.387410 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387388 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-tls\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.387833 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.387833 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387675 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-client-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.387833 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387739 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvvqw\" (UniqueName: \"kubernetes.io/projected/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-kube-api-access-kvvqw\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.388251 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.387774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-audit-log\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.388558 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.388402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-metrics-server-audit-profiles\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.489997 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.489960 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-metrics-server-audit-profiles\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490137 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-client-certs\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490137 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-tls\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490137 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490234 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-client-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvvqw\" (UniqueName: \"kubernetes.io/projected/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-kube-api-access-kvvqw\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-audit-log\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490678 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490655 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-audit-log\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.490822 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.490752 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.491077 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.491057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-metrics-server-audit-profiles\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.492398 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.492377 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-client-certs\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.492517 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.492499 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-client-ca-bundle\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.492600 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.492583 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-secret-metrics-server-tls\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.498948 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.498931 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvvqw\" (UniqueName: \"kubernetes.io/projected/cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50-kube-api-access-kvvqw\") pod \"metrics-server-d4d665b8-ds9hd\" (UID: \"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50\") " pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.561961 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.561878 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:33.700420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.700352 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d4d665b8-ds9hd"] Jul 09 22:15:33.701588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.701560 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:33.703340 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:33.703315 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbb998a_a4d5_4a7b_b2d0_e93bdcaa5e50.slice/crio-01679157e2b974ef35caad0d8395b63423d385ee10b7641cbee9c7618ecdba13 WatchSource:0}: Error finding container 01679157e2b974ef35caad0d8395b63423d385ee10b7641cbee9c7618ecdba13: Status 404 returned error can't find the container with id 01679157e2b974ef35caad0d8395b63423d385ee10b7641cbee9c7618ecdba13 Jul 09 22:15:33.724641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.722319 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:15:33.727200 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.727130 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.730216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730195 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 22:15:33.730413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730393 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 22:15:33.730413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730442 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 22:15:33.730413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730453 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 22:15:33.730646 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730393 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 22:15:33.730731 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730663 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 22:15:33.730880 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730824 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 22:15:33.730880 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.730859 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l6k92\"" Jul 09 22:15:33.736482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.736007 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 22:15:33.737869 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.737849 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:15:33.774064 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.774045 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:33.793176 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793154 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793264 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793183 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb9b4\" (UniqueName: \"kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793264 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793211 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793264 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793371 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793330 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793371 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.793446 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.793380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.891539 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.891462 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:33.894571 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb9b4\" (UniqueName: \"kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894640 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894746 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.894876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.894772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.895452 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.895431 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.895556 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.895431 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.895615 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.895552 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.895874 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.895857 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.897242 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.897224 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.897327 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.897309 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.903526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.903502 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb9b4\" (UniqueName: \"kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4\") pod \"console-db45465dd-2kq9m\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:33.992404 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:33.992375 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:34.039529 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.039504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:34.146836 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.146805 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:34.206171 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.206135 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:15:34.211445 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:15:34.211412 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f8b4e4_5d0b_4285_b92c_c8bfade6fbfe.slice/crio-9c58966f49462c2e583da99037dd4089570598cf2b4b5c4bd5b603399081b8ad WatchSource:0}: Error finding container 9c58966f49462c2e583da99037dd4089570598cf2b4b5c4bd5b603399081b8ad: Status 404 returned error can't find the container with id 9c58966f49462c2e583da99037dd4089570598cf2b4b5c4bd5b603399081b8ad Jul 09 22:15:34.214607 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.214510 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:15:34.245248 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.245210 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:34.285825 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.285788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" event={"ID":"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50","Type":"ContainerStarted","Data":"01679157e2b974ef35caad0d8395b63423d385ee10b7641cbee9c7618ecdba13"} Jul 09 22:15:34.287490 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:34.287461 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db45465dd-2kq9m" event={"ID":"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe","Type":"ContainerStarted","Data":"9c58966f49462c2e583da99037dd4089570598cf2b4b5c4bd5b603399081b8ad"} Jul 09 22:15:35.215537 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:35.215497 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:35.606532 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:35.606416 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:35.711888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:35.711853 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:36.301016 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:36.300984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"64ba25d14a84d8178e14014f45783a230990887980dd67438b03f5d69659e57e"} Jul 09 22:15:36.301207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:36.301070 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"08759775b42ab90278b794e3c17d5871cfbc23c81c428ea0643b9597fd7ebe34"} Jul 09 22:15:36.301207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:36.301115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"818a49a684e47f497f915235a7fc23d0d8ba139f1239b327faa5d3b3c3915a9f"} Jul 09 22:15:36.302747 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:36.302716 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" event={"ID":"cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50","Type":"ContainerStarted","Data":"8d011af9001492c9fe2b7c3da292b3c9f968d669311e01bb092f7e59240cee6b"} Jul 09 22:15:36.324647 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:36.324595 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" podStartSLOduration=1.153207111 podStartE2EDuration="3.324576272s" totalImagesPullingTime="2.171369161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:36.321252922 +0000 UTC m=+305.012826986" watchObservedRunningTime="2026-07-09 22:15:36.324576272 +0000 UTC m=+305.016150327" Jul 09 22:15:37.310253 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:37.310210 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"c60383779bb1bd825147df8f2c9473db80b509db902d3e4d6221ce9b7d234559"} Jul 09 22:15:37.310641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:37.310284 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"55f3225cd316fd486c14a1114a5a0eb85433cd8d99d0ec28f42fcd9da0bdc408"} Jul 09 22:15:37.311788 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:37.311762 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db45465dd-2kq9m" event={"ID":"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe","Type":"ContainerStarted","Data":"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4"} Jul 09 22:15:37.334568 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:37.334523 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-db45465dd-2kq9m" podStartSLOduration=1.377369818 podStartE2EDuration="4.334508909s" totalImagesPullingTime="2.957139091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:37.332511726 +0000 UTC m=+306.024085783" watchObservedRunningTime="2026-07-09 22:15:37.334508909 +0000 UTC m=+306.026082962" Jul 09 22:15:38.185994 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:38.185974 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:15:38.318984 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:38.318937 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35691cf3-c17e-429d-9f0f-6347619e03e2","Type":"ContainerStarted","Data":"908ed26574852b1d1063d86740949953d680779b06dde04292fce20e3fb7ed24"} Jul 09 22:15:38.349644 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:38.349520 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.682134291 podStartE2EDuration="9.349501773s" totalImagesPullingTime="4.655883779s" totalInitContainerRuntime="11.483703ms" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:38.348984239 +0000 UTC m=+307.040558293" watchObservedRunningTime="2026-07-09 22:15:38.349501773 +0000 UTC m=+307.041075826" Jul 09 22:15:39.316297 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:39.316265 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:39.361575 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:39.361546 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:15:39.397629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:39.397596 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:39.774656 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:39.774624 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:40.359112 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:40.359074 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:40.420650 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:40.420622 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:44.040054 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:44.040020 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:15:50.613696 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:50.613660 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:50.699064 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:50.699035 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:50.867159 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:50.867077 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:50.967715 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:50.967685 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:15:53.562680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:53.562643 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:53.563182 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:53.562727 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:15:55.391697 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:55.391664 2575 generic.go:356] "Generic (PLEG): container finished" podID="48090bc2-a6c4-4412-826a-cc4c65e24085" containerID="52ee52e3628d5be4886cf1ba75a142f3f568250a544b561434a33109958f0842" exitCode=0 Jul 09 22:15:55.392060 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:55.391737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" event={"ID":"48090bc2-a6c4-4412-826a-cc4c65e24085","Type":"ContainerDied","Data":"52ee52e3628d5be4886cf1ba75a142f3f568250a544b561434a33109958f0842"} Jul 09 22:15:55.392110 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:55.392066 2575 scope.go:117] "RemoveContainer" containerID="52ee52e3628d5be4886cf1ba75a142f3f568250a544b561434a33109958f0842" Jul 09 22:15:56.397203 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:15:56.397171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-8btqh" event={"ID":"48090bc2-a6c4-4412-826a-cc4c65e24085","Type":"ContainerStarted","Data":"4fb73c73e51c54b237d44f96d76faaed80b122cf6ab552360bbd084cb9047caf"} Jul 09 22:16:04.382640 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.382585 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-db45465dd-2kq9m" podUID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" containerName="console" containerID="cri-o://3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4" gracePeriod=15 Jul 09 22:16:04.625631 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.625604 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:16:04.664604 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664580 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664614 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664647 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664678 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664718 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb9b4\" (UniqueName: \"kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664788 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.664964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.664834 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config\") pod \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\" (UID: \"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe\") " Jul 09 22:16:04.665156 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.665127 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:16:04.665283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.665161 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config" (OuterVolumeSpecName: "console-config") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:16:04.665283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.665171 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca" (OuterVolumeSpecName: "service-ca") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:16:04.665283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.665271 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:16:04.667149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.667119 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:16:04.667149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.667134 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:16:04.667149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.667129 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4" (OuterVolumeSpecName: "kube-api-access-jb9b4") pod "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" (UID: "c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe"). InnerVolumeSpecName "kube-api-access-jb9b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:16:04.765831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765801 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-config\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.765831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765826 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-trusted-ca-bundle\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.765831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765836 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.766051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765845 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb9b4\" (UniqueName: \"kubernetes.io/projected/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-kube-api-access-jb9b4\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.766051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765854 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-serving-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.766051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765864 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-console-oauth-config\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:04.766051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:04.765872 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe-oauth-serving-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:16:05.431521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.431485 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" containerID="3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4" exitCode=0 Jul 09 22:16:05.431521 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.431522 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-db45465dd-2kq9m" Jul 09 22:16:05.431965 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.431559 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db45465dd-2kq9m" event={"ID":"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe","Type":"ContainerDied","Data":"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4"} Jul 09 22:16:05.431965 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.431603 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-db45465dd-2kq9m" event={"ID":"c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe","Type":"ContainerDied","Data":"9c58966f49462c2e583da99037dd4089570598cf2b4b5c4bd5b603399081b8ad"} Jul 09 22:16:05.431965 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.431634 2575 scope.go:117] "RemoveContainer" containerID="3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4" Jul 09 22:16:05.439941 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.439918 2575 scope.go:117] "RemoveContainer" containerID="3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4" Jul 09 22:16:05.440180 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:16:05.440158 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4\": container with ID starting with 3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4 not found: ID does not exist" containerID="3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4" Jul 09 22:16:05.440251 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.440187 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4"} err="failed to get container status \"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4\": rpc error: code = NotFound desc = could not find container \"3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4\": container with ID starting with 3da6f868358921f1f452ed694b284d578b9d5009414550215c31ea3c08002bf4 not found: ID does not exist" Jul 09 22:16:05.449547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.449499 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:16:05.453208 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.453190 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-db45465dd-2kq9m"] Jul 09 22:16:05.630746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.630714 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:05.703138 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:05.703060 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:07.192103 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:07.192061 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" path="/var/lib/kubelet/pods/c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe/volumes" Jul 09 22:16:13.567994 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:13.567965 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:16:13.571652 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:13.571625 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d4d665b8-ds9hd" Jul 09 22:16:24.978087 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:24.978001 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:25.250886 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:25.250805 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.037838 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.037804 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.144676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.144646 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.368949 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.368859 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:16:45.369188 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.369175 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" containerName="console" Jul 09 22:16:45.369227 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.369189 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" containerName="console" Jul 09 22:16:45.369265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.369241 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f8b4e4-5d0b-4285-b92c-c8bfade6fbfe" containerName="console" Jul 09 22:16:45.373564 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.373540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.378928 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.378907 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 22:16:45.379140 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.379119 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 22:16:45.379269 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.379250 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 22:16:45.379439 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.379423 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 22:16:45.379559 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.378998 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l6k92\"" Jul 09 22:16:45.379612 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.379131 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 22:16:45.379911 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.379878 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 22:16:45.380318 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.380302 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 22:16:45.384005 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.383982 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 22:16:45.385467 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.385448 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:16:45.394773 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394751 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.394883 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.394883 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394811 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.394883 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.395085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.395085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.394969 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.395085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.395015 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hstkh\" (UniqueName: \"kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.435211 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.435184 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.495837 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.495982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hstkh\" (UniqueName: \"kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.495982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.495982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495943 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.495982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495959 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.495982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.495976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.496254 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.496047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.496772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.496748 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.496875 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.496774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.496875 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.496809 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.496875 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.496822 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.498344 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.498322 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.498428 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.498413 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.503972 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.503954 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hstkh\" (UniqueName: \"kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh\") pod \"console-659f4c58f-xwh8b\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.536559 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.536536 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.637637 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.637534 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:45.686156 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.686114 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:45.817418 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:45.817392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:16:45.819470 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:16:45.819442 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca530fc_50aa_4f53_ade6_a138e385af40.slice/crio-041c3a7590101edcf549c94fde931d34c51bfd953751ac9fe7c57685c373cce3 WatchSource:0}: Error finding container 041c3a7590101edcf549c94fde931d34c51bfd953751ac9fe7c57685c373cce3: Status 404 returned error can't find the container with id 041c3a7590101edcf549c94fde931d34c51bfd953751ac9fe7c57685c373cce3 Jul 09 22:16:46.100472 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.100435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:16:46.102670 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.102653 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4361956b-0edf-45fb-90e6-ab52b1b3286b-metrics-certs\") pod \"network-metrics-daemon-pwcd7\" (UID: \"4361956b-0edf-45fb-90e6-ab52b1b3286b\") " pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:16:46.158502 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.158471 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:46.212607 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.212583 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:46.289919 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.289865 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jtrvn\"" Jul 09 22:16:46.297467 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.297448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pwcd7" Jul 09 22:16:46.421999 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.421974 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pwcd7"] Jul 09 22:16:46.424399 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:16:46.424374 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4361956b_0edf_45fb_90e6_ab52b1b3286b.slice/crio-c079f62ea6e6d62e9423d953a46b35cae66458050263b6322f87f068c6a372bc WatchSource:0}: Error finding container c079f62ea6e6d62e9423d953a46b35cae66458050263b6322f87f068c6a372bc: Status 404 returned error can't find the container with id c079f62ea6e6d62e9423d953a46b35cae66458050263b6322f87f068c6a372bc Jul 09 22:16:46.588803 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.588769 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwcd7" event={"ID":"4361956b-0edf-45fb-90e6-ab52b1b3286b","Type":"ContainerStarted","Data":"c079f62ea6e6d62e9423d953a46b35cae66458050263b6322f87f068c6a372bc"} Jul 09 22:16:46.590344 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.590316 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659f4c58f-xwh8b" event={"ID":"1ca530fc-50aa-4f53-ade6-a138e385af40","Type":"ContainerStarted","Data":"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda"} Jul 09 22:16:46.590475 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.590356 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659f4c58f-xwh8b" event={"ID":"1ca530fc-50aa-4f53-ade6-a138e385af40","Type":"ContainerStarted","Data":"041c3a7590101edcf549c94fde931d34c51bfd953751ac9fe7c57685c373cce3"} Jul 09 22:16:46.611318 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:46.611278 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-659f4c58f-xwh8b" podStartSLOduration=1.611264302 podStartE2EDuration="1.611264302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:16:46.610301623 +0000 UTC m=+375.301875669" watchObservedRunningTime="2026-07-09 22:16:46.611264302 +0000 UTC m=+375.302838354" Jul 09 22:16:47.594948 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:47.594923 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwcd7" event={"ID":"4361956b-0edf-45fb-90e6-ab52b1b3286b","Type":"ContainerStarted","Data":"dce6df3022f51ba166c302ae1e0a95ec6a9bc2128029bf235a8521bd544d1052"} Jul 09 22:16:47.834159 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:47.834081 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:48.078143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.078109 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:48.600806 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.600772 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pwcd7" event={"ID":"4361956b-0edf-45fb-90e6-ab52b1b3286b","Type":"ContainerStarted","Data":"1ec822f3f17345e7ba1e4eef12c145807c40f79d8b6b582c491127d39ac05661"} Jul 09 22:16:48.619852 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.619797 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pwcd7" podStartSLOduration=375.620006579 podStartE2EDuration="6m16.619783485s" totalImagesPullingTime="999.776906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:16:48.617743604 +0000 UTC m=+377.309317670" watchObservedRunningTime="2026-07-09 22:16:48.619783485 +0000 UTC m=+377.311357537" Jul 09 22:16:48.712969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.712943 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:48.774654 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.774630 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:48.833764 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:48.833744 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:55.686778 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:55.686748 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:55.687153 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:55.686812 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:55.691737 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:55.691716 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:56.635910 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:56.635868 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:16:56.799219 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:56.799189 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:56.898955 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:56.898928 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:56.952515 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:56.952493 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:57.067126 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:57.067097 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:16:57.168046 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:16:57.167983 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:05.617833 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:05.617770 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:05.720029 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:05.719996 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:05.790332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:05.790303 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:26.277195 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:26.277158 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:26.378113 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:26.378082 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:29.385948 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:29.385914 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:17:29.487876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:17:29.487849 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:18:01.041207 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:18:01.041125 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:12.176656 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:12.176623 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:12.951194 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:12.951159 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:15.799306 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:15.799271 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:15.903599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:15.903572 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:17.298865 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:17.298831 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:17.402129 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:17.402095 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:17.611200 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:17.611128 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:17.736130 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:17.736102 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:19.498057 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.498020 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:19.679844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.679816 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:19.768725 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.768652 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-487jn"] Jul 09 22:19:19.771869 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.771854 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.774547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.774520 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 22:19:19.774665 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.774646 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 22:19:19.775848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.775832 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mwzsm\"" Jul 09 22:19:19.784208 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.784186 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-487jn"] Jul 09 22:19:19.788991 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.788970 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-774jk\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-kube-api-access-774jk\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.789086 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.789016 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.889847 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.889816 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-774jk\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-kube-api-access-774jk\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.889989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.889858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.898529 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.898499 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:19.898623 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:19.898581 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-774jk\" (UniqueName: \"kubernetes.io/projected/3ca02334-a05b-4f16-b9a0-bbbd1237ae41-kube-api-access-774jk\") pod \"cert-manager-cainjector-559cdd8c87-487jn\" (UID: \"3ca02334-a05b-4f16-b9a0-bbbd1237ae41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:20.080339 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:20.080262 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" Jul 09 22:19:20.205733 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:20.205704 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-487jn"] Jul 09 22:19:20.208474 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:19:20.208445 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ca02334_a05b_4f16_b9a0_bbbd1237ae41.slice/crio-7d177243fea46aec5952eb99639469bc634dd9e2158467431dc013fc15446517 WatchSource:0}: Error finding container 7d177243fea46aec5952eb99639469bc634dd9e2158467431dc013fc15446517: Status 404 returned error can't find the container with id 7d177243fea46aec5952eb99639469bc634dd9e2158467431dc013fc15446517 Jul 09 22:19:21.197760 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:21.197725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" event={"ID":"3ca02334-a05b-4f16-b9a0-bbbd1237ae41","Type":"ContainerStarted","Data":"7d177243fea46aec5952eb99639469bc634dd9e2158467431dc013fc15446517"} Jul 09 22:19:22.586666 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:22.586613 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:22.687418 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:22.687387 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:24.212617 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:24.212582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" event={"ID":"3ca02334-a05b-4f16-b9a0-bbbd1237ae41","Type":"ContainerStarted","Data":"2a8eaf91d041de8cc6ade0c96a18c7883b5ffc99a4d263e7d78bfa56488cc342"} Jul 09 22:19:24.229603 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:24.229550 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-487jn" podStartSLOduration=1.732492906 podStartE2EDuration="5.22953194s" totalImagesPullingTime="3.497039034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:24.228943864 +0000 UTC m=+532.920517920" watchObservedRunningTime="2026-07-09 22:19:24.22953194 +0000 UTC m=+532.921105994" Jul 09 22:19:24.511752 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:24.511646 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:24.576100 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:24.576071 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:27.260414 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:27.260375 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:27.694572 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:27.694533 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:28.442437 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.442409 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wj254"] Jul 09 22:19:28.445400 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.445384 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.448161 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.448140 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qtzq2\"" Jul 09 22:19:28.454399 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.454378 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wj254"] Jul 09 22:19:28.563958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.563935 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-bound-sa-token\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.564082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.564010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw7rs\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-kube-api-access-nw7rs\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.606210 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.606186 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:28.665312 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.665290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw7rs\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-kube-api-access-nw7rs\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.665393 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.665322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-bound-sa-token\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.674125 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.674104 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-bound-sa-token\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.674333 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.674313 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw7rs\" (UniqueName: \"kubernetes.io/projected/f0f5fe9d-3fd3-439b-89e5-02ae6415690e-kube-api-access-nw7rs\") pod \"cert-manager-55b4d954d5-wj254\" (UID: \"f0f5fe9d-3fd3-439b-89e5-02ae6415690e\") " pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.707373 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.707306 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:28.754451 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.754429 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wj254" Jul 09 22:19:28.878075 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:28.877929 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wj254"] Jul 09 22:19:28.880251 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:19:28.880224 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0f5fe9d_3fd3_439b_89e5_02ae6415690e.slice/crio-02d05e7fe7c8637ca41b21f35eefbd619a5c32d6cddd1ab3b7499a72072304b0 WatchSource:0}: Error finding container 02d05e7fe7c8637ca41b21f35eefbd619a5c32d6cddd1ab3b7499a72072304b0: Status 404 returned error can't find the container with id 02d05e7fe7c8637ca41b21f35eefbd619a5c32d6cddd1ab3b7499a72072304b0 Jul 09 22:19:29.231752 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:29.231717 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wj254" event={"ID":"f0f5fe9d-3fd3-439b-89e5-02ae6415690e","Type":"ContainerStarted","Data":"8ffb2b378977579ec147973d58b130b894bb8d5b8460daa52eb8c402479f279e"} Jul 09 22:19:29.231946 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:29.231762 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wj254" event={"ID":"f0f5fe9d-3fd3-439b-89e5-02ae6415690e","Type":"ContainerStarted","Data":"02d05e7fe7c8637ca41b21f35eefbd619a5c32d6cddd1ab3b7499a72072304b0"} Jul 09 22:19:29.249638 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:29.249590 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wj254" podStartSLOduration=1.249575246 podStartE2EDuration="1.249575246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:29.248780131 +0000 UTC m=+537.940354189" watchObservedRunningTime="2026-07-09 22:19:29.249575246 +0000 UTC m=+537.941149298" Jul 09 22:19:34.730142 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:34.730113 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:34.835116 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:34.835071 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:35.357321 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:35.357291 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:35.460701 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:35.460659 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:36.824273 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:36.824237 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:37.197744 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:37.197715 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:40.608660 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:40.608624 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:41.744322 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:41.744289 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:42.452259 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:42.452227 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:42.515011 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:42.514977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:52.997516 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:52.997479 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:53.099078 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:53.099045 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:53.632326 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:53.632295 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:53.960982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:53.960954 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.065058 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.065027 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.126969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.126929 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9"] Jul 09 22:19:54.130928 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.130880 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.135250 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.135229 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 22:19:54.136026 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.136011 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 22:19:54.136114 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.136086 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 22:19:54.136280 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.136259 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 22:19:54.136443 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.136368 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8n8ll\"" Jul 09 22:19:54.150567 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.150541 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9"] Jul 09 22:19:54.244651 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.244582 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.283317 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.283294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.283435 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.283332 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkts\" (UniqueName: \"kubernetes.io/projected/ebb7f9c8-3c6d-431b-a998-e12c29925513-kube-api-access-8fkts\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.283435 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.283375 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.283435 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.283393 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-tmp\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.283542 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.283468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-manifests\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.344634 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.344601 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.383868 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.383845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkts\" (UniqueName: \"kubernetes.io/projected/ebb7f9c8-3c6d-431b-a998-e12c29925513-kube-api-access-8fkts\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.384000 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.383875 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.384000 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.383923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-tmp\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.384000 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.383966 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-manifests\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.384121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.384031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.384392 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.384371 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-manifests\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.386298 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.386267 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebb7f9c8-3c6d-431b-a998-e12c29925513-tmp\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.386411 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.386355 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.386411 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.386353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebb7f9c8-3c6d-431b-a998-e12c29925513-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.392166 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.392146 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkts\" (UniqueName: \"kubernetes.io/projected/ebb7f9c8-3c6d-431b-a998-e12c29925513-kube-api-access-8fkts\") pod \"opendatahub-operator-controller-manager-5cf875db85-8s6z9\" (UID: \"ebb7f9c8-3c6d-431b-a998-e12c29925513\") " pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.441639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.441607 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:19:54.444697 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.444668 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.545750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.545723 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.572776 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.572753 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9"] Jul 09 22:19:54.575370 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:19:54.575343 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb7f9c8_3c6d_431b_a998_e12c29925513.slice/crio-f4780fbf4f366cafa2c75d02f0caa22a509aa3ba60d9876e2cf3ef4a452e1531 WatchSource:0}: Error finding container f4780fbf4f366cafa2c75d02f0caa22a509aa3ba60d9876e2cf3ef4a452e1531: Status 404 returned error can't find the container with id f4780fbf4f366cafa2c75d02f0caa22a509aa3ba60d9876e2cf3ef4a452e1531 Jul 09 22:19:54.645672 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.645643 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:54.757722 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:54.757692 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:19:55.337728 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:55.337681 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" event={"ID":"ebb7f9c8-3c6d-431b-a998-e12c29925513","Type":"ContainerStarted","Data":"f4780fbf4f366cafa2c75d02f0caa22a509aa3ba60d9876e2cf3ef4a452e1531"} Jul 09 22:19:58.352840 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:58.352800 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" event={"ID":"ebb7f9c8-3c6d-431b-a998-e12c29925513","Type":"ContainerStarted","Data":"3b6ccb4e872f0ae5fddba0f0341cb3ceb1f731a53dbc546ebc5f2da5afec64bc"} Jul 09 22:19:59.359473 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:59.359438 2575 generic.go:356] "Generic (PLEG): container finished" podID="ebb7f9c8-3c6d-431b-a998-e12c29925513" containerID="3b6ccb4e872f0ae5fddba0f0341cb3ceb1f731a53dbc546ebc5f2da5afec64bc" exitCode=0 Jul 09 22:19:59.359876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:19:59.359502 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" event={"ID":"ebb7f9c8-3c6d-431b-a998-e12c29925513","Type":"ContainerDied","Data":"3b6ccb4e872f0ae5fddba0f0341cb3ceb1f731a53dbc546ebc5f2da5afec64bc"} Jul 09 22:20:00.365350 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:00.365306 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" event={"ID":"ebb7f9c8-3c6d-431b-a998-e12c29925513","Type":"ContainerStarted","Data":"4e1157ac27e64d5791ba227446952dc38e843ca44b0330b8c2e8b53c7e3a8237"} Jul 09 22:20:00.365726 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:00.365388 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:20:00.403685 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:00.403621 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" podStartSLOduration=1.9643834359999999 podStartE2EDuration="6.403602503s" totalImagesPullingTime="3.351968526s" totalInitContainerRuntime="1.087250541s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:00.401527738 +0000 UTC m=+569.093101790" watchObservedRunningTime="2026-07-09 22:20:00.403602503 +0000 UTC m=+569.095176557" Jul 09 22:20:01.471587 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:01.471555 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:01.577183 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:01.577150 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:11.371707 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:11.371659 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cf875db85-8s6z9" Jul 09 22:20:12.993379 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:12.993335 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:13.144060 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.144027 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:13.500045 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.500011 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:13.623079 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.623045 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:13.765984 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.765907 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6"] Jul 09 22:20:13.771321 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.771301 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.775473 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.775450 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 22:20:13.775606 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.775511 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 22:20:13.775606 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.775572 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 22:20:13.775731 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.775606 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 22:20:13.775876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.775857 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mgmgt\"" Jul 09 22:20:13.799126 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.799104 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6"] Jul 09 22:20:13.830489 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.830466 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvf8\" (UniqueName: \"kubernetes.io/projected/2592be26-7d07-4f5a-902d-214aefcd8cec-kube-api-access-vmvf8\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.830569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.830513 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2592be26-7d07-4f5a-902d-214aefcd8cec-tmp\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.830615 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.830565 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2592be26-7d07-4f5a-902d-214aefcd8cec-tls-certs\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.906078 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.906051 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:13.930948 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.930923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2592be26-7d07-4f5a-902d-214aefcd8cec-tls-certs\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.931080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.930986 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvf8\" (UniqueName: \"kubernetes.io/projected/2592be26-7d07-4f5a-902d-214aefcd8cec-kube-api-access-vmvf8\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.931080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.931067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2592be26-7d07-4f5a-902d-214aefcd8cec-tmp\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.933265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.933246 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2592be26-7d07-4f5a-902d-214aefcd8cec-tls-certs\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.933372 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.933333 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2592be26-7d07-4f5a-902d-214aefcd8cec-tmp\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:13.940599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:13.940574 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvf8\" (UniqueName: \"kubernetes.io/projected/2592be26-7d07-4f5a-902d-214aefcd8cec-kube-api-access-vmvf8\") pod \"kube-auth-proxy-5ff74c44b9-d9wr6\" (UID: \"2592be26-7d07-4f5a-902d-214aefcd8cec\") " pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:14.009299 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.009271 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:14.081245 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.081184 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" Jul 09 22:20:14.106457 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.106429 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:14.215593 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.215570 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6"] Jul 09 22:20:14.217476 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:20:14.217448 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2592be26_7d07_4f5a_902d_214aefcd8cec.slice/crio-f29533fa638cf63d3f81a2598e4d91f46a6f9e04bbdad7ddd6c594d41411a6df WatchSource:0}: Error finding container f29533fa638cf63d3f81a2598e4d91f46a6f9e04bbdad7ddd6c594d41411a6df: Status 404 returned error can't find the container with id f29533fa638cf63d3f81a2598e4d91f46a6f9e04bbdad7ddd6c594d41411a6df Jul 09 22:20:14.426555 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.426523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" event={"ID":"2592be26-7d07-4f5a-902d-214aefcd8cec","Type":"ContainerStarted","Data":"f29533fa638cf63d3f81a2598e4d91f46a6f9e04bbdad7ddd6c594d41411a6df"} Jul 09 22:20:14.620852 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.620804 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:14.864854 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:14.864767 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:15.138109 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:15.138005 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:15.235041 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:15.235013 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:15.463642 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:15.463609 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:16.267492 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:16.267430 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:16.433639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:16.433604 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:17.443075 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:17.443034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" event={"ID":"2592be26-7d07-4f5a-902d-214aefcd8cec","Type":"ContainerStarted","Data":"e277beb63bf9f67e8e2f3af3e1ef904e842bf86fd2cd7a70d28f8530942bce0e"} Jul 09 22:20:17.461081 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:17.461029 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5ff74c44b9-d9wr6" podStartSLOduration=1.791313289 podStartE2EDuration="4.461014402s" totalImagesPullingTime="2.669701113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:17.459344903 +0000 UTC m=+586.150918962" watchObservedRunningTime="2026-07-09 22:20:17.461014402 +0000 UTC m=+586.152588482" Jul 09 22:20:21.268633 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.268603 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9"] Jul 09 22:20:21.270760 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.270745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.274126 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.274104 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 22:20:21.274505 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.274492 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 22:20:21.274558 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.274511 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 22:20:21.274788 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.274768 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hz88g\"" Jul 09 22:20:21.275077 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.275063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 22:20:21.295057 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295058 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcdlq\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-kube-api-access-kcdlq\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295165 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295335 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/941d1cf0-7d9c-4033-b423-93f02753bd43-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.295517 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.295389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.298371 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.298350 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9"] Jul 09 22:20:21.333410 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.333386 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:21.396704 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396704 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396707 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396727 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcdlq\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-kube-api-access-kcdlq\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396769 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396792 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/941d1cf0-7d9c-4033-b423-93f02753bd43-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.396964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.396876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.397485 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.397443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.399328 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.399280 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.399540 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.399524 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/941d1cf0-7d9c-4033-b423-93f02753bd43-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.399641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.399620 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.399711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.399659 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/941d1cf0-7d9c-4033-b423-93f02753bd43-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.405780 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.405752 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcdlq\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-kube-api-access-kcdlq\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.405980 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.405962 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941d1cf0-7d9c-4033-b423-93f02753bd43-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-62zk9\" (UID: \"941d1cf0-7d9c-4033-b423-93f02753bd43\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.432768 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.432744 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:21.579505 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.579422 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:21.650989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.650924 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:21.746565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:21.746541 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9"] Jul 09 22:20:21.749304 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:20:21.749278 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941d1cf0_7d9c_4033_b423_93f02753bd43.slice/crio-1a42da8d66775fc546e9f0393d55de2f14111c73f997006497c955f8c94991d8 WatchSource:0}: Error finding container 1a42da8d66775fc546e9f0393d55de2f14111c73f997006497c955f8c94991d8: Status 404 returned error can't find the container with id 1a42da8d66775fc546e9f0393d55de2f14111c73f997006497c955f8c94991d8 Jul 09 22:20:22.024143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:22.024115 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:22.143680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:22.143629 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:22.482932 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:22.482869 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" event={"ID":"941d1cf0-7d9c-4033-b423-93f02753bd43","Type":"ContainerStarted","Data":"1a42da8d66775fc546e9f0393d55de2f14111c73f997006497c955f8c94991d8"} Jul 09 22:20:24.055760 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.055725 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 22:20:24.056028 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.055791 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 22:20:24.496840 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.496794 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" event={"ID":"941d1cf0-7d9c-4033-b423-93f02753bd43","Type":"ContainerStarted","Data":"937b6478f9d849b5203e724b430f39d440ee8fece666bff2f077420e93bea521"} Jul 09 22:20:24.497296 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.497271 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:24.505094 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.505058 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:24.529080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.529020 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" podStartSLOduration=1.225365318 podStartE2EDuration="3.528999174s" totalImagesPullingTime="2.303633856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:24.525528245 +0000 UTC m=+593.217102299" watchObservedRunningTime="2026-07-09 22:20:24.528999174 +0000 UTC m=+593.220573230" Jul 09 22:20:24.617667 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.617637 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:24.916392 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:24.916362 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:25.134197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:25.134163 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:25.264007 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:25.263926 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:25.503335 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:25.503310 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-62zk9" Jul 09 22:20:32.347261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.347231 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:20:32.347874 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.347415 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:20:32.349008 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.348989 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:20:32.349227 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.349209 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:20:32.353390 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.353370 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:20:32.353733 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:32.353718 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:20:41.544214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.544181 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-88d96b4f6-7l5kr"] Jul 09 22:20:41.546738 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.546711 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.575979 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.575952 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88d96b4f6-7l5kr"] Jul 09 22:20:41.617687 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.617663 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:41.664149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-oauth-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-oauth-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-service-ca\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664363 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664265 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-trusted-ca-bundle\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664363 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664288 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5w7\" (UniqueName: \"kubernetes.io/projected/e6197cf4-169c-4213-80ed-203cd6ba90ae-kube-api-access-xt5w7\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664363 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.664469 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.664377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.686529 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.686505 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:41.764751 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-service-ca\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.764846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764766 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-trusted-ca-bundle\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.764846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5w7\" (UniqueName: \"kubernetes.io/projected/e6197cf4-169c-4213-80ed-203cd6ba90ae-kube-api-access-xt5w7\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.764846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.764846 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765042 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764861 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-oauth-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765042 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.764932 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-oauth-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765684 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.765638 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765684 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.765670 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-oauth-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.765686 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-service-ca\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.765964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.765946 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6197cf4-169c-4213-80ed-203cd6ba90ae-trusted-ca-bundle\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.767323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.767296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-oauth-config\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.767406 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.767388 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6197cf4-169c-4213-80ed-203cd6ba90ae-console-serving-cert\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.775791 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.775770 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5w7\" (UniqueName: \"kubernetes.io/projected/e6197cf4-169c-4213-80ed-203cd6ba90ae-kube-api-access-xt5w7\") pod \"console-88d96b4f6-7l5kr\" (UID: \"e6197cf4-169c-4213-80ed-203cd6ba90ae\") " pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.856158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.856111 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:41.983803 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:41.983779 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88d96b4f6-7l5kr"] Jul 09 22:20:41.986280 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:20:41.986253 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6197cf4_169c_4213_80ed_203cd6ba90ae.slice/crio-a20987e13b193f518ac79eac4b0a9bda60aff276fc4ccd199c1489312a6ad498 WatchSource:0}: Error finding container a20987e13b193f518ac79eac4b0a9bda60aff276fc4ccd199c1489312a6ad498: Status 404 returned error can't find the container with id a20987e13b193f518ac79eac4b0a9bda60aff276fc4ccd199c1489312a6ad498 Jul 09 22:20:42.572797 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:42.572757 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88d96b4f6-7l5kr" event={"ID":"e6197cf4-169c-4213-80ed-203cd6ba90ae","Type":"ContainerStarted","Data":"cd66e381b3538bc944fce4f5b6f10cf0ee2b725bf7356df0ccd5bee987a3dc80"} Jul 09 22:20:42.573169 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:42.572807 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88d96b4f6-7l5kr" event={"ID":"e6197cf4-169c-4213-80ed-203cd6ba90ae","Type":"ContainerStarted","Data":"a20987e13b193f518ac79eac4b0a9bda60aff276fc4ccd199c1489312a6ad498"} Jul 09 22:20:42.593732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:42.593682 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-88d96b4f6-7l5kr" podStartSLOduration=1.593663413 podStartE2EDuration="1.593663413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:42.593446298 +0000 UTC m=+611.285020353" watchObservedRunningTime="2026-07-09 22:20:42.593663413 +0000 UTC m=+611.285237469" Jul 09 22:20:51.857107 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:51.857058 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:51.857519 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:51.857126 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:51.861672 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:51.861652 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:52.625856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:52.625826 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-88d96b4f6-7l5kr" Jul 09 22:20:52.675163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:52.675132 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:20:52.873845 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:52.873814 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:20:52.964180 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:20:52.964154 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:09.991066 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:09.991035 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:14.510187 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.510155 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:14.602880 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.602850 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:14.640217 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.640183 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:14.646603 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.646583 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:14.649578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.649552 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 22:21:14.649715 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.649553 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-l9t7d\"" Jul 09 22:21:14.649715 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.649556 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 22:21:14.654369 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.654326 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:14.689113 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.689084 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:14.745250 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.745217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q9vf\" (UniqueName: \"kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf\") pod \"kuadrant-operator-catalog-vl2mj\" (UID: \"890fa881-9b5b-410a-8695-cf2b2437719d\") " pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:14.845682 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.845604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q9vf\" (UniqueName: \"kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf\") pod \"kuadrant-operator-catalog-vl2mj\" (UID: \"890fa881-9b5b-410a-8695-cf2b2437719d\") " pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:14.857823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.857794 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q9vf\" (UniqueName: \"kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf\") pod \"kuadrant-operator-catalog-vl2mj\" (UID: \"890fa881-9b5b-410a-8695-cf2b2437719d\") " pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:14.959174 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:14.959139 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:15.019548 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.019480 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:15.110638 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.110618 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:15.112725 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:21:15.112701 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890fa881_9b5b_410a_8695_cf2b2437719d.slice/crio-b069b2f46cbbd5f6b13b6ac50687b4dac0ef9411b4ebffe46abe7dbffa66807c WatchSource:0}: Error finding container b069b2f46cbbd5f6b13b6ac50687b4dac0ef9411b4ebffe46abe7dbffa66807c: Status 404 returned error can't find the container with id b069b2f46cbbd5f6b13b6ac50687b4dac0ef9411b4ebffe46abe7dbffa66807c Jul 09 22:21:15.114533 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.114516 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:21:15.218162 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.218137 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtrgc"] Jul 09 22:21:15.221405 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.221390 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:15.230087 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.230060 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtrgc"] Jul 09 22:21:15.248732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.248695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvz4q\" (UniqueName: \"kubernetes.io/projected/e466afd8-dc85-4df7-9b0a-87c5be8955bc-kube-api-access-zvz4q\") pod \"kuadrant-operator-catalog-vtrgc\" (UID: \"e466afd8-dc85-4df7-9b0a-87c5be8955bc\") " pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:15.350103 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.350077 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvz4q\" (UniqueName: \"kubernetes.io/projected/e466afd8-dc85-4df7-9b0a-87c5be8955bc-kube-api-access-zvz4q\") pod \"kuadrant-operator-catalog-vtrgc\" (UID: \"e466afd8-dc85-4df7-9b0a-87c5be8955bc\") " pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:15.359664 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.359634 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvz4q\" (UniqueName: \"kubernetes.io/projected/e466afd8-dc85-4df7-9b0a-87c5be8955bc-kube-api-access-zvz4q\") pod \"kuadrant-operator-catalog-vtrgc\" (UID: \"e466afd8-dc85-4df7-9b0a-87c5be8955bc\") " pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:15.531581 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.531547 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:15.677257 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.677197 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtrgc"] Jul 09 22:21:15.679229 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:21:15.679203 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode466afd8_dc85_4df7_9b0a_87c5be8955bc.slice/crio-470f32220fed15912669f730bb63497ebad05faead575996fa76a8bf2eb335ad WatchSource:0}: Error finding container 470f32220fed15912669f730bb63497ebad05faead575996fa76a8bf2eb335ad: Status 404 returned error can't find the container with id 470f32220fed15912669f730bb63497ebad05faead575996fa76a8bf2eb335ad Jul 09 22:21:15.721222 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.721198 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" event={"ID":"e466afd8-dc85-4df7-9b0a-87c5be8955bc","Type":"ContainerStarted","Data":"470f32220fed15912669f730bb63497ebad05faead575996fa76a8bf2eb335ad"} Jul 09 22:21:15.722344 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:15.722321 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" event={"ID":"890fa881-9b5b-410a-8695-cf2b2437719d","Type":"ContainerStarted","Data":"b069b2f46cbbd5f6b13b6ac50687b4dac0ef9411b4ebffe46abe7dbffa66807c"} Jul 09 22:21:17.697724 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.697684 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-659f4c58f-xwh8b" podUID="1ca530fc-50aa-4f53-ade6-a138e385af40" containerName="console" containerID="cri-o://18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda" gracePeriod=15 Jul 09 22:21:17.735356 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.735329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" event={"ID":"e466afd8-dc85-4df7-9b0a-87c5be8955bc","Type":"ContainerStarted","Data":"2beb348ab79fa6a57220def8e07359055d10fe8408d88958a72b8ab1a275e0f5"} Jul 09 22:21:17.736792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.736768 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" event={"ID":"890fa881-9b5b-410a-8695-cf2b2437719d","Type":"ContainerStarted","Data":"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509"} Jul 09 22:21:17.736792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.736777 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" podUID="890fa881-9b5b-410a-8695-cf2b2437719d" containerName="registry-server" containerID="cri-o://34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509" gracePeriod=2 Jul 09 22:21:17.756106 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.756068 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" podStartSLOduration=1.193054719 podStartE2EDuration="2.756053339s" totalImagesPullingTime="1.56299862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:17.754004094 +0000 UTC m=+646.445578148" watchObservedRunningTime="2026-07-09 22:21:17.756053339 +0000 UTC m=+646.447627391" Jul 09 22:21:17.773538 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.773495 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" podStartSLOduration=1.6472945220000001 podStartE2EDuration="3.773482559s" totalImagesPullingTime="2.126188037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:17.77284056 +0000 UTC m=+646.464414633" watchObservedRunningTime="2026-07-09 22:21:17.773482559 +0000 UTC m=+646.465056611" Jul 09 22:21:17.976881 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.976854 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:21:17.983498 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:17.983477 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:18.075461 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075437 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q9vf\" (UniqueName: \"kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf\") pod \"890fa881-9b5b-410a-8695-cf2b2437719d\" (UID: \"890fa881-9b5b-410a-8695-cf2b2437719d\") " Jul 09 22:21:18.075588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075486 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075588 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075518 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075658 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075627 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075716 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075702 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075755 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075732 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075755 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075751 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hstkh\" (UniqueName: \"kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.075856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075768 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config\") pod \"1ca530fc-50aa-4f53-ade6-a138e385af40\" (UID: \"1ca530fc-50aa-4f53-ade6-a138e385af40\") " Jul 09 22:21:18.076019 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075959 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config" (OuterVolumeSpecName: "console-config") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:21:18.076102 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.075963 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:21:18.076102 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.076067 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:21:18.076102 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.076077 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-console-config\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.076259 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.076130 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:21:18.077714 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.077683 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf" (OuterVolumeSpecName: "kube-api-access-6q9vf") pod "890fa881-9b5b-410a-8695-cf2b2437719d" (UID: "890fa881-9b5b-410a-8695-cf2b2437719d"). InnerVolumeSpecName "kube-api-access-6q9vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:21:18.077811 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.077713 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:21:18.077811 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.077786 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:21:18.077969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.077936 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh" (OuterVolumeSpecName: "kube-api-access-hstkh") pod "1ca530fc-50aa-4f53-ade6-a138e385af40" (UID: "1ca530fc-50aa-4f53-ade6-a138e385af40"). InnerVolumeSpecName "kube-api-access-hstkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:21:18.176702 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176673 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-serving-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176702 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176698 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176711 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hstkh\" (UniqueName: \"kubernetes.io/projected/1ca530fc-50aa-4f53-ade6-a138e385af40-kube-api-access-hstkh\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176724 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ca530fc-50aa-4f53-ade6-a138e385af40-console-oauth-config\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176736 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q9vf\" (UniqueName: \"kubernetes.io/projected/890fa881-9b5b-410a-8695-cf2b2437719d-kube-api-access-6q9vf\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176746 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-trusted-ca-bundle\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.176873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.176757 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ca530fc-50aa-4f53-ade6-a138e385af40-oauth-serving-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:21:18.742103 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.742070 2575 generic.go:356] "Generic (PLEG): container finished" podID="1ca530fc-50aa-4f53-ade6-a138e385af40" containerID="18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda" exitCode=0 Jul 09 22:21:18.742579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.742117 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659f4c58f-xwh8b" Jul 09 22:21:18.742579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.742161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659f4c58f-xwh8b" event={"ID":"1ca530fc-50aa-4f53-ade6-a138e385af40","Type":"ContainerDied","Data":"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda"} Jul 09 22:21:18.742579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.742219 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659f4c58f-xwh8b" event={"ID":"1ca530fc-50aa-4f53-ade6-a138e385af40","Type":"ContainerDied","Data":"041c3a7590101edcf549c94fde931d34c51bfd953751ac9fe7c57685c373cce3"} Jul 09 22:21:18.742579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.742253 2575 scope.go:117] "RemoveContainer" containerID="18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda" Jul 09 22:21:18.743628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.743488 2575 generic.go:356] "Generic (PLEG): container finished" podID="890fa881-9b5b-410a-8695-cf2b2437719d" containerID="34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509" exitCode=0 Jul 09 22:21:18.743628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.743529 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" Jul 09 22:21:18.743628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.743572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" event={"ID":"890fa881-9b5b-410a-8695-cf2b2437719d","Type":"ContainerDied","Data":"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509"} Jul 09 22:21:18.743628 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.743613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vl2mj" event={"ID":"890fa881-9b5b-410a-8695-cf2b2437719d","Type":"ContainerDied","Data":"b069b2f46cbbd5f6b13b6ac50687b4dac0ef9411b4ebffe46abe7dbffa66807c"} Jul 09 22:21:18.753206 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.753190 2575 scope.go:117] "RemoveContainer" containerID="18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda" Jul 09 22:21:18.753498 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:21:18.753472 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda\": container with ID starting with 18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda not found: ID does not exist" containerID="18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda" Jul 09 22:21:18.753562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.753498 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda"} err="failed to get container status \"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda\": rpc error: code = NotFound desc = could not find container \"18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda\": container with ID starting with 18784bcd028526843b179841ddaf96f771d65b5dc49b6321a7e9a4852a95ffda not found: ID does not exist" Jul 09 22:21:18.753562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.753514 2575 scope.go:117] "RemoveContainer" containerID="34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509" Jul 09 22:21:18.761379 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.761362 2575 scope.go:117] "RemoveContainer" containerID="34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509" Jul 09 22:21:18.761601 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:21:18.761586 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509\": container with ID starting with 34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509 not found: ID does not exist" containerID="34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509" Jul 09 22:21:18.761645 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.761609 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509"} err="failed to get container status \"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509\": rpc error: code = NotFound desc = could not find container \"34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509\": container with ID starting with 34830c182b99b67e0f1afe8ed438d615c93c0f96117f6ec38dbbaaae2ba7b509 not found: ID does not exist" Jul 09 22:21:18.781945 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.781919 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:21:18.792725 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.792702 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-659f4c58f-xwh8b"] Jul 09 22:21:18.812244 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.812220 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:18.817958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:18.817940 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vl2mj"] Jul 09 22:21:19.191731 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:19.191700 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ca530fc-50aa-4f53-ade6-a138e385af40" path="/var/lib/kubelet/pods/1ca530fc-50aa-4f53-ade6-a138e385af40/volumes" Jul 09 22:21:19.192044 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:19.192031 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="890fa881-9b5b-410a-8695-cf2b2437719d" path="/var/lib/kubelet/pods/890fa881-9b5b-410a-8695-cf2b2437719d/volumes" Jul 09 22:21:25.532508 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:25.532472 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:25.532969 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:25.532535 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:25.554953 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:25.554927 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:25.798866 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:25.798795 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vtrgc" Jul 09 22:21:42.091420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:42.091387 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:47.278500 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278472 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws"] Jul 09 22:21:47.278859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278812 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890fa881-9b5b-410a-8695-cf2b2437719d" containerName="registry-server" Jul 09 22:21:47.278859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278824 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="890fa881-9b5b-410a-8695-cf2b2437719d" containerName="registry-server" Jul 09 22:21:47.278859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278834 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca530fc-50aa-4f53-ade6-a138e385af40" containerName="console" Jul 09 22:21:47.278859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278840 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca530fc-50aa-4f53-ade6-a138e385af40" containerName="console" Jul 09 22:21:47.279074 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278920 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca530fc-50aa-4f53-ade6-a138e385af40" containerName="console" Jul 09 22:21:47.279074 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.278936 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="890fa881-9b5b-410a-8695-cf2b2437719d" containerName="registry-server" Jul 09 22:21:47.283313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.283295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.285998 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.285978 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ngbqm\"" Jul 09 22:21:47.300794 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.300770 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws"] Jul 09 22:21:47.434281 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.434240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.434281 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.434281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7skq\" (UniqueName: \"kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.535708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.535627 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.535708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.535669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7skq\" (UniqueName: \"kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.536025 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.536007 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.546664 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.546634 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7skq\" (UniqueName: \"kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.599172 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.599144 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:47.753249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.753227 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws"] Jul 09 22:21:47.755682 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:21:47.755644 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169e10c2_402e_481f_b2a2_284ffe17df33.slice/crio-fb3810261e0a8e3480de18078a41440083aad01a9e082370eb3be36cdd7b7840 WatchSource:0}: Error finding container fb3810261e0a8e3480de18078a41440083aad01a9e082370eb3be36cdd7b7840: Status 404 returned error can't find the container with id fb3810261e0a8e3480de18078a41440083aad01a9e082370eb3be36cdd7b7840 Jul 09 22:21:47.875610 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:47.875549 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" event={"ID":"169e10c2-402e-481f-b2a2-284ffe17df33","Type":"ContainerStarted","Data":"fb3810261e0a8e3480de18078a41440083aad01a9e082370eb3be36cdd7b7840"} Jul 09 22:21:48.309218 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:48.309185 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:50.388234 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.388199 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59"] Jul 09 22:21:50.392080 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.392058 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:50.396155 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.396131 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g9khx\"" Jul 09 22:21:50.412606 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.412586 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59"] Jul 09 22:21:50.465024 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.464995 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8cjw\" (UniqueName: \"kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw\") pod \"limitador-operator-controller-manager-859d7b78bd-vlr59\" (UID: \"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:50.565863 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.565837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8cjw\" (UniqueName: \"kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw\") pod \"limitador-operator-controller-manager-859d7b78bd-vlr59\" (UID: \"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:50.574701 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.574677 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8cjw\" (UniqueName: \"kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw\") pod \"limitador-operator-controller-manager-859d7b78bd-vlr59\" (UID: \"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:50.675821 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.675792 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:50.705234 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.705210 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:50.838763 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.838737 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59"] Jul 09 22:21:50.841513 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:21:50.841483 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8985d539_d35b_4ad4_ba1c_cab4c1eeec5b.slice/crio-905cd92f8caf58e1a5b095a5b64b044918afa7f5de51964e9837ee0f49b563a9 WatchSource:0}: Error finding container 905cd92f8caf58e1a5b095a5b64b044918afa7f5de51964e9837ee0f49b563a9: Status 404 returned error can't find the container with id 905cd92f8caf58e1a5b095a5b64b044918afa7f5de51964e9837ee0f49b563a9 Jul 09 22:21:50.890997 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.890970 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" event={"ID":"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b","Type":"ContainerStarted","Data":"905cd92f8caf58e1a5b095a5b64b044918afa7f5de51964e9837ee0f49b563a9"} Jul 09 22:21:50.993995 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:50.993887 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:51.319591 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:51.319506 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:52.722999 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:52.722967 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:52.903646 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:52.903584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" event={"ID":"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b","Type":"ContainerStarted","Data":"5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb"} Jul 09 22:21:52.903861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:52.903690 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:21:52.921787 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:52.921741 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" podStartSLOduration=1.023776392 podStartE2EDuration="2.92172476s" totalImagesPullingTime="1.897948368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:52.921214941 +0000 UTC m=+681.612789007" watchObservedRunningTime="2026-07-09 22:21:52.92172476 +0000 UTC m=+681.613298793" Jul 09 22:21:55.092362 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.092336 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:55.338145 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.338038 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:55.437842 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.437815 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:55.921250 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.921216 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" event={"ID":"169e10c2-402e-481f-b2a2-284ffe17df33","Type":"ContainerStarted","Data":"a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172"} Jul 09 22:21:55.921447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.921296 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:21:55.944211 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:55.944160 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" podStartSLOduration=1.689569809 podStartE2EDuration="8.944142541s" totalImagesPullingTime="7.254572732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:55.942371758 +0000 UTC m=+684.633945811" watchObservedRunningTime="2026-07-09 22:21:55.944142541 +0000 UTC m=+684.635716594" Jul 09 22:21:57.341702 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:57.341674 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:57.674337 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:57.674304 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:57.929532 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:57.929453 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:59.700320 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:59.700286 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:21:59.798878 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:21:59.798850 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:03.910958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:03.910930 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:06.928198 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:06.928166 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:08.276601 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:08.276566 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:08.485282 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:08.485251 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:09.268772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.268745 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws"] Jul 09 22:22:09.268988 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.268966 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" containerName="manager" containerID="cri-o://a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172" gracePeriod=2 Jul 09 22:22:09.287661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.287636 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws"] Jul 09 22:22:09.326904 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.326863 2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"169e10c2-402e-481f-b2a2-284ffe17df33\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-sbvws\": pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.329193 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.329173 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59"] Jul 09 22:22:09.329362 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.329342 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" containerName="manager" containerID="cri-o://5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb" gracePeriod=2 Jul 09 22:22:09.342911 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.342773 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.360465 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.360439 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59"] Jul 09 22:22:09.364123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364098 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:09.364483 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364467 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" containerName="manager" Jul 09 22:22:09.364547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364485 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" containerName="manager" Jul 09 22:22:09.364547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364507 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" containerName="manager" Jul 09 22:22:09.364547 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364515 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" containerName="manager" Jul 09 22:22:09.364686 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364573 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" containerName="manager" Jul 09 22:22:09.364686 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.364585 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" containerName="manager" Jul 09 22:22:09.367706 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.367688 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.403827 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.403798 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:09.423130 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.423105 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv"] Jul 09 22:22:09.425917 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.425870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9h2n\" (UniqueName: \"kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.426019 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.425939 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.426265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.426248 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.436391 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.436273 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.438905 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.438866 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.441278 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.441252 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.445462 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.445440 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.458857 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.458831 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv"] Jul 09 22:22:09.527510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.527447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnxbz\" (UniqueName: \"kubernetes.io/projected/939f0861-a88d-4e00-ba28-0cfda0243fb3-kube-api-access-bnxbz\") pod \"limitador-operator-controller-manager-859d7b78bd-466pv\" (UID: \"939f0861-a88d-4e00-ba28-0cfda0243fb3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.527510 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.527488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9h2n\" (UniqueName: \"kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.527687 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.527576 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.528068 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.528049 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.572402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.572370 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9h2n\" (UniqueName: \"kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xcvkv\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.576604 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.576583 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.579249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.579227 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.579491 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.579470 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.583983 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.583958 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.586554 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.586536 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.588958 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.588932 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.627853 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.627837 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8cjw\" (UniqueName: \"kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw\") pod \"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b\" (UID: \"8985d539-d35b-4ad4-ba1c-cab4c1eeec5b\") " Jul 09 22:22:09.627957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.627875 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7skq\" (UniqueName: \"kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq\") pod \"169e10c2-402e-481f-b2a2-284ffe17df33\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " Jul 09 22:22:09.627957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.627920 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume\") pod \"169e10c2-402e-481f-b2a2-284ffe17df33\" (UID: \"169e10c2-402e-481f-b2a2-284ffe17df33\") " Jul 09 22:22:09.628110 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.628096 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnxbz\" (UniqueName: \"kubernetes.io/projected/939f0861-a88d-4e00-ba28-0cfda0243fb3-kube-api-access-bnxbz\") pod \"limitador-operator-controller-manager-859d7b78bd-466pv\" (UID: \"939f0861-a88d-4e00-ba28-0cfda0243fb3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.628409 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.628387 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "169e10c2-402e-481f-b2a2-284ffe17df33" (UID: "169e10c2-402e-481f-b2a2-284ffe17df33"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:22:09.629823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.629802 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw" (OuterVolumeSpecName: "kube-api-access-z8cjw") pod "8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" (UID: "8985d539-d35b-4ad4-ba1c-cab4c1eeec5b"). InnerVolumeSpecName "kube-api-access-z8cjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:22:09.629962 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.629925 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq" (OuterVolumeSpecName: "kube-api-access-r7skq") pod "169e10c2-402e-481f-b2a2-284ffe17df33" (UID: "169e10c2-402e-481f-b2a2-284ffe17df33"). InnerVolumeSpecName "kube-api-access-r7skq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:22:09.644232 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.644214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnxbz\" (UniqueName: \"kubernetes.io/projected/939f0861-a88d-4e00-ba28-0cfda0243fb3-kube-api-access-bnxbz\") pod \"limitador-operator-controller-manager-859d7b78bd-466pv\" (UID: \"939f0861-a88d-4e00-ba28-0cfda0243fb3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.710952 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.710934 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:09.728879 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.728862 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8cjw\" (UniqueName: \"kubernetes.io/projected/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b-kube-api-access-z8cjw\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:09.728964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.728880 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7skq\" (UniqueName: \"kubernetes.io/projected/169e10c2-402e-481f-b2a2-284ffe17df33-kube-api-access-r7skq\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:09.728964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.728906 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169e10c2-402e-481f-b2a2-284ffe17df33-extensions-socket-volume\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:09.736824 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.736803 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.920730 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.920707 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv"] Jul 09 22:22:09.922357 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:22:09.922329 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939f0861_a88d_4e00_ba28_0cfda0243fb3.slice/crio-18b9cfcdb6b27cea3f86d94e5dcb83e2cf2fcfffeb17bc2656ca861ced6835df WatchSource:0}: Error finding container 18b9cfcdb6b27cea3f86d94e5dcb83e2cf2fcfffeb17bc2656ca861ced6835df: Status 404 returned error can't find the container with id 18b9cfcdb6b27cea3f86d94e5dcb83e2cf2fcfffeb17bc2656ca861ced6835df Jul 09 22:22:09.922505 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.922485 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:09.922875 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:22:09.922858 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e06413_f02f_4653_bd7a_b3d969a616da.slice/crio-b557a7fb0a49a09141900c615c2a70551c7d563bbb607de50c1935c6ac8c8a9c WatchSource:0}: Error finding container b557a7fb0a49a09141900c615c2a70551c7d563bbb607de50c1935c6ac8c8a9c: Status 404 returned error can't find the container with id b557a7fb0a49a09141900c615c2a70551c7d563bbb607de50c1935c6ac8c8a9c Jul 09 22:22:09.993861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.993833 2575 generic.go:356] "Generic (PLEG): container finished" podID="169e10c2-402e-481f-b2a2-284ffe17df33" containerID="a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172" exitCode=0 Jul 09 22:22:09.993970 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.993883 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.993970 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.993934 2575 scope.go:117] "RemoveContainer" containerID="a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172" Jul 09 22:22:09.995802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.995775 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" event={"ID":"939f0861-a88d-4e00-ba28-0cfda0243fb3","Type":"ContainerStarted","Data":"18b9cfcdb6b27cea3f86d94e5dcb83e2cf2fcfffeb17bc2656ca861ced6835df"} Jul 09 22:22:09.996175 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.995887 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:09.997370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.997348 2575 generic.go:356] "Generic (PLEG): container finished" podID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" containerID="5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb" exitCode=0 Jul 09 22:22:09.997464 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.997386 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:09.998557 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:09.998425 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:09.999201 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.999181 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" event={"ID":"42e06413-f02f-4653-bd7a-b3d969a616da","Type":"ContainerStarted","Data":"b557a7fb0a49a09141900c615c2a70551c7d563bbb607de50c1935c6ac8c8a9c"} Jul 09 22:22:09.999324 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:09.999312 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:10.002621 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.002591 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:10.003398 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.003383 2575 scope.go:117] "RemoveContainer" containerID="a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172" Jul 09 22:22:10.003704 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.003686 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172\": container with ID starting with a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172 not found: ID does not exist" containerID="a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172" Jul 09 22:22:10.003781 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.003714 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172"} err="failed to get container status \"a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172\": rpc error: code = NotFound desc = could not find container \"a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172\": container with ID starting with a6a1b1995c548c54247ef285d3924f2bfaf5ea221d54cafdb0a0f68e66305172 not found: ID does not exist" Jul 09 22:22:10.003781 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.003735 2575 scope.go:117] "RemoveContainer" containerID="5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb" Jul 09 22:22:10.011067 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.011050 2575 scope.go:117] "RemoveContainer" containerID="5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb" Jul 09 22:22:10.011312 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.011296 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb\": container with ID starting with 5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb not found: ID does not exist" containerID="5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb" Jul 09 22:22:10.011372 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.011318 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb"} err="failed to get container status \"5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb\": rpc error: code = NotFound desc = could not find container \"5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb\": container with ID starting with 5c2dadb18997b22b3554d9946930d24948fbe7909933de6e118ec446d9c45fdb not found: ID does not exist" Jul 09 22:22:10.038043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:10.038003 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" podStartSLOduration=1.037991621 podStartE2EDuration="1.037991621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:10.03625639 +0000 UTC m=+698.727830445" watchObservedRunningTime="2026-07-09 22:22:10.037991621 +0000 UTC m=+698.729565673" Jul 09 22:22:10.042478 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.042453 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:10.045002 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.044960 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:10.078781 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.078751 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbvws\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbvws" Jul 09 22:22:10.084319 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:10.084298 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vlr59\" is forbidden: User \"system:node:ip-10-0-130-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-127.ec2.internal' and this object" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlr59" Jul 09 22:22:11.006086 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.006052 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" event={"ID":"939f0861-a88d-4e00-ba28-0cfda0243fb3","Type":"ContainerStarted","Data":"3475adada746bf9f51128cccf2b1d416842194f0fed1cf0a240da57816948002"} Jul 09 22:22:11.008535 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.008509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" event={"ID":"42e06413-f02f-4653-bd7a-b3d969a616da","Type":"ContainerStarted","Data":"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898"} Jul 09 22:22:11.191820 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.191788 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="169e10c2-402e-481f-b2a2-284ffe17df33" path="/var/lib/kubelet/pods/169e10c2-402e-481f-b2a2-284ffe17df33/volumes" Jul 09 22:22:11.192136 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.192122 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8985d539-d35b-4ad4-ba1c-cab4c1eeec5b" path="/var/lib/kubelet/pods/8985d539-d35b-4ad4-ba1c-cab4c1eeec5b/volumes" Jul 09 22:22:11.464782 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.464737 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" podStartSLOduration=2.464718076 podStartE2EDuration="2.464718076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:10.076198098 +0000 UTC m=+698.767772166" watchObservedRunningTime="2026-07-09 22:22:11.464718076 +0000 UTC m=+700.156292130" Jul 09 22:22:11.466428 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.466405 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:22:11.471378 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.471361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.486162 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.485880 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:22:11.545512 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.545477 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.545667 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.545545 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmdj\" (UniqueName: \"kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.646772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.646721 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmdj\" (UniqueName: \"kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.646976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.646811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.647179 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.647163 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.663299 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.663273 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmdj\" (UniqueName: \"kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj\") pod \"kuadrant-operator-controller-manager-86878c769f-wlrj6\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.781460 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.781380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:11.924576 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:11.924541 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:22:11.927635 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:22:11.927607 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683e5009_8fa4_4979_9d95_bec3d6264354.slice/crio-4c3d41324db1e91ecfe5fdf74e84cacbb2521c04a02baa2f90b12004af187f93 WatchSource:0}: Error finding container 4c3d41324db1e91ecfe5fdf74e84cacbb2521c04a02baa2f90b12004af187f93: Status 404 returned error can't find the container with id 4c3d41324db1e91ecfe5fdf74e84cacbb2521c04a02baa2f90b12004af187f93 Jul 09 22:22:12.015573 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:12.015540 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" event={"ID":"683e5009-8fa4-4979-9d95-bec3d6264354","Type":"ContainerStarted","Data":"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b"} Jul 09 22:22:12.015945 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:12.015773 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:12.015945 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:12.015805 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" event={"ID":"683e5009-8fa4-4979-9d95-bec3d6264354","Type":"ContainerStarted","Data":"4c3d41324db1e91ecfe5fdf74e84cacbb2521c04a02baa2f90b12004af187f93"} Jul 09 22:22:12.036089 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:12.035990 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" podStartSLOduration=1.035975475 podStartE2EDuration="1.035975475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:12.034279476 +0000 UTC m=+700.725853530" watchObservedRunningTime="2026-07-09 22:22:12.035975475 +0000 UTC m=+700.727549528" Jul 09 22:22:12.945358 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:12.945321 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:13.052311 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:13.052279 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:15.375811 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:15.375780 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:15.480247 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:15.480201 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:21.310271 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:21.310244 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:21.376260 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:21.376229 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:22.017530 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:22.017498 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-466pv" Jul 09 22:22:22.017734 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:22.017717 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:23.023157 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.023126 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:22:23.114241 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.114207 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:23.114429 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.114405 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" podUID="42e06413-f02f-4653-bd7a-b3d969a616da" containerName="manager" containerID="cri-o://aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898" gracePeriod=10 Jul 09 22:22:23.353939 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.353916 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:23.445136 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.445107 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9h2n\" (UniqueName: \"kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n\") pod \"42e06413-f02f-4653-bd7a-b3d969a616da\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " Jul 09 22:22:23.445329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.445220 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume\") pod \"42e06413-f02f-4653-bd7a-b3d969a616da\" (UID: \"42e06413-f02f-4653-bd7a-b3d969a616da\") " Jul 09 22:22:23.445599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.445574 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "42e06413-f02f-4653-bd7a-b3d969a616da" (UID: "42e06413-f02f-4653-bd7a-b3d969a616da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:22:23.447407 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.447373 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n" (OuterVolumeSpecName: "kube-api-access-h9h2n") pod "42e06413-f02f-4653-bd7a-b3d969a616da" (UID: "42e06413-f02f-4653-bd7a-b3d969a616da"). InnerVolumeSpecName "kube-api-access-h9h2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:22:23.546313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.546241 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42e06413-f02f-4653-bd7a-b3d969a616da-extensions-socket-volume\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:23.546313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:23.546265 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9h2n\" (UniqueName: \"kubernetes.io/projected/42e06413-f02f-4653-bd7a-b3d969a616da-kube-api-access-h9h2n\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:24.082791 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.082757 2575 generic.go:356] "Generic (PLEG): container finished" podID="42e06413-f02f-4653-bd7a-b3d969a616da" containerID="aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898" exitCode=0 Jul 09 22:22:24.083261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.082799 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" Jul 09 22:22:24.083261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.082843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" event={"ID":"42e06413-f02f-4653-bd7a-b3d969a616da","Type":"ContainerDied","Data":"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898"} Jul 09 22:22:24.083261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.082912 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv" event={"ID":"42e06413-f02f-4653-bd7a-b3d969a616da","Type":"ContainerDied","Data":"b557a7fb0a49a09141900c615c2a70551c7d563bbb607de50c1935c6ac8c8a9c"} Jul 09 22:22:24.083261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.082941 2575 scope.go:117] "RemoveContainer" containerID="aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898" Jul 09 22:22:24.092132 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.092116 2575 scope.go:117] "RemoveContainer" containerID="aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898" Jul 09 22:22:24.092370 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:22:24.092353 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898\": container with ID starting with aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898 not found: ID does not exist" containerID="aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898" Jul 09 22:22:24.092417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.092378 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898"} err="failed to get container status \"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898\": rpc error: code = NotFound desc = could not find container \"aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898\": container with ID starting with aaaf63ddc955c58b7812345282ab938b496e7dbd73d284445eb2e59203b73898 not found: ID does not exist" Jul 09 22:22:24.116031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.116012 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:24.124152 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:24.124133 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xcvkv"] Jul 09 22:22:25.192157 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:25.192126 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e06413-f02f-4653-bd7a-b3d969a616da" path="/var/lib/kubelet/pods/42e06413-f02f-4653-bd7a-b3d969a616da/volumes" Jul 09 22:22:28.598234 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:28.598201 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:28.711213 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:28.711184 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:33.364840 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:33.364809 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:33.461579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:33.461553 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:33.581315 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:33.581287 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:35.774796 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:35.774762 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:35.836580 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:35.836549 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:39.583281 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:39.583230 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:39.922789 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:39.922753 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:41.475403 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.475374 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:41.488197 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.488168 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:41.488526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.488514 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e06413-f02f-4653-bd7a-b3d969a616da" containerName="manager" Jul 09 22:22:41.488576 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.488528 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e06413-f02f-4653-bd7a-b3d969a616da" containerName="manager" Jul 09 22:22:41.488612 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.488590 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e06413-f02f-4653-bd7a-b3d969a616da" containerName="manager" Jul 09 22:22:41.491469 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.491450 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.494220 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.494202 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fkpk4\"" Jul 09 22:22:41.494332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.494271 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 22:22:41.497801 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.497776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfmb\" (UniqueName: \"kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.497923 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.497884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.507633 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.507610 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:41.578129 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.578096 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:41.598557 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.598527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.598708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.598586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfmb\" (UniqueName: \"kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.599213 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.599190 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.602869 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.602849 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:41.608361 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.608332 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfmb\" (UniqueName: \"kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb\") pod \"limitador-limitador-64c94fd64f-6cgh8\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.709154 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.709120 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:41.802063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.801998 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:41.936939 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:41.936885 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:41.939208 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:22:41.939182 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a84d2b_0db6_4359_9f7a_773da63763b7.slice/crio-69453727872b916708fa407980c6b56e1f07af9cb6f4d4164ec5e968f4bfc7bf WatchSource:0}: Error finding container 69453727872b916708fa407980c6b56e1f07af9cb6f4d4164ec5e968f4bfc7bf: Status 404 returned error can't find the container with id 69453727872b916708fa407980c6b56e1f07af9cb6f4d4164ec5e968f4bfc7bf Jul 09 22:22:42.165855 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:42.165823 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" event={"ID":"d4a84d2b-0db6-4359-9f7a-773da63763b7","Type":"ContainerStarted","Data":"69453727872b916708fa407980c6b56e1f07af9cb6f4d4164ec5e968f4bfc7bf"} Jul 09 22:22:42.343818 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:42.343786 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:42.440091 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:42.440019 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:45.182993 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:45.182952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" event={"ID":"d4a84d2b-0db6-4359-9f7a-773da63763b7","Type":"ContainerStarted","Data":"211265458f9d51195f85eb144fd6736337972cdf96650f8dd11d341aa53b4433"} Jul 09 22:22:45.183427 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:45.183144 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:45.201880 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:45.201834 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" podStartSLOduration=1.575611099 podStartE2EDuration="4.201819924s" totalImagesPullingTime="2.626208825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:45.20119658 +0000 UTC m=+733.892770632" watchObservedRunningTime="2026-07-09 22:22:45.201819924 +0000 UTC m=+733.893393977" Jul 09 22:22:55.039507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:55.039471 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:56.189017 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:56.188990 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:56.510580 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:56.510494 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:56.510752 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:56.510709 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" podUID="d4a84d2b-0db6-4359-9f7a-773da63763b7" containerName="limitador" containerID="cri-o://211265458f9d51195f85eb144fd6736337972cdf96650f8dd11d341aa53b4433" gracePeriod=30 Jul 09 22:22:57.235003 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.234971 2575 generic.go:356] "Generic (PLEG): container finished" podID="d4a84d2b-0db6-4359-9f7a-773da63763b7" containerID="211265458f9d51195f85eb144fd6736337972cdf96650f8dd11d341aa53b4433" exitCode=0 Jul 09 22:22:57.235359 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.235039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" event={"ID":"d4a84d2b-0db6-4359-9f7a-773da63763b7","Type":"ContainerDied","Data":"211265458f9d51195f85eb144fd6736337972cdf96650f8dd11d341aa53b4433"} Jul 09 22:22:57.451347 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.451322 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:57.535248 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.535183 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvfmb\" (UniqueName: \"kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb\") pod \"d4a84d2b-0db6-4359-9f7a-773da63763b7\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " Jul 09 22:22:57.535248 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.535228 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file\") pod \"d4a84d2b-0db6-4359-9f7a-773da63763b7\" (UID: \"d4a84d2b-0db6-4359-9f7a-773da63763b7\") " Jul 09 22:22:57.535589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.535568 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file" (OuterVolumeSpecName: "config-file") pod "d4a84d2b-0db6-4359-9f7a-773da63763b7" (UID: "d4a84d2b-0db6-4359-9f7a-773da63763b7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:22:57.537218 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.537196 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb" (OuterVolumeSpecName: "kube-api-access-kvfmb") pod "d4a84d2b-0db6-4359-9f7a-773da63763b7" (UID: "d4a84d2b-0db6-4359-9f7a-773da63763b7"). InnerVolumeSpecName "kube-api-access-kvfmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:22:57.636534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.636500 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvfmb\" (UniqueName: \"kubernetes.io/projected/d4a84d2b-0db6-4359-9f7a-773da63763b7-kube-api-access-kvfmb\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:57.636534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.636528 2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d4a84d2b-0db6-4359-9f7a-773da63763b7-config-file\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:22:57.640501 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.640466 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:57.732009 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:57.731978 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:58.063947 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.063917 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hsn56"] Jul 09 22:22:58.064270 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.064254 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a84d2b-0db6-4359-9f7a-773da63763b7" containerName="limitador" Jul 09 22:22:58.064359 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.064271 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a84d2b-0db6-4359-9f7a-773da63763b7" containerName="limitador" Jul 09 22:22:58.064417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.064367 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a84d2b-0db6-4359-9f7a-773da63763b7" containerName="limitador" Jul 09 22:22:58.067418 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.067402 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.070230 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.070207 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 22:22:58.070354 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.070207 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 22:22:58.071612 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.071592 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4fkdz\"" Jul 09 22:22:58.071768 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.071592 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 22:22:58.077376 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.077357 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hsn56"] Jul 09 22:22:58.103765 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.103731 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:22:58.140935 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.140888 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9krnl\" (UniqueName: \"kubernetes.io/projected/e52bd149-2915-4219-92f4-ef20307a59b3-kube-api-access-9krnl\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.141048 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.140999 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e52bd149-2915-4219-92f4-ef20307a59b3-data\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.241420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241394 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" Jul 09 22:22:58.241420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241403 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9krnl\" (UniqueName: \"kubernetes.io/projected/e52bd149-2915-4219-92f4-ef20307a59b3-kube-api-access-9krnl\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.242082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e52bd149-2915-4219-92f4-ef20307a59b3-data\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.242082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6cgh8" event={"ID":"d4a84d2b-0db6-4359-9f7a-773da63763b7","Type":"ContainerDied","Data":"69453727872b916708fa407980c6b56e1f07af9cb6f4d4164ec5e968f4bfc7bf"} Jul 09 22:22:58.242082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241580 2575 scope.go:117] "RemoveContainer" containerID="211265458f9d51195f85eb144fd6736337972cdf96650f8dd11d341aa53b4433" Jul 09 22:22:58.242082 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.241860 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e52bd149-2915-4219-92f4-ef20307a59b3-data\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.250828 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.250805 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9krnl\" (UniqueName: \"kubernetes.io/projected/e52bd149-2915-4219-92f4-ef20307a59b3-kube-api-access-9krnl\") pod \"postgres-69fb4594cf-hsn56\" (UID: \"e52bd149-2915-4219-92f4-ef20307a59b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.269630 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.269610 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:58.273182 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.273163 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6cgh8"] Jul 09 22:22:58.381684 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.381631 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:22:58.510282 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:58.510259 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hsn56"] Jul 09 22:22:58.512359 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:22:58.512335 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52bd149_2915_4219_92f4_ef20307a59b3.slice/crio-c385d1f06256080368915e3d0a0bb6f003a0eeec79951c3a3cfb98b15a81b6ea WatchSource:0}: Error finding container c385d1f06256080368915e3d0a0bb6f003a0eeec79951c3a3cfb98b15a81b6ea: Status 404 returned error can't find the container with id c385d1f06256080368915e3d0a0bb6f003a0eeec79951c3a3cfb98b15a81b6ea Jul 09 22:22:59.191470 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:59.191438 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a84d2b-0db6-4359-9f7a-773da63763b7" path="/var/lib/kubelet/pods/d4a84d2b-0db6-4359-9f7a-773da63763b7/volumes" Jul 09 22:22:59.247297 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:22:59.247267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" event={"ID":"e52bd149-2915-4219-92f4-ef20307a59b3","Type":"ContainerStarted","Data":"c385d1f06256080368915e3d0a0bb6f003a0eeec79951c3a3cfb98b15a81b6ea"} Jul 09 22:23:04.273075 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:04.273044 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" event={"ID":"e52bd149-2915-4219-92f4-ef20307a59b3","Type":"ContainerStarted","Data":"2be514c9b655f2d38a702233e8f29868d59a21f770cf35bf81d67a1f00c1537c"} Jul 09 22:23:04.273447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:04.273171 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:23:04.292083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:04.292034 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" podStartSLOduration=1.48194757 podStartE2EDuration="6.292019436s" totalImagesPullingTime="4.810071866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:04.2914427 +0000 UTC m=+752.983016755" watchObservedRunningTime="2026-07-09 22:23:04.292019436 +0000 UTC m=+752.983593488" Jul 09 22:23:10.307670 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:10.307644 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hsn56" Jul 09 22:23:10.623642 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:10.623560 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:10.842304 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:10.842276 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:16.850287 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.850242 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:16.879220 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.879187 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:16.908998 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.908968 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:16.909171 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.909095 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:16.911991 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.911967 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 22:23:16.912119 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.912054 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dpv7x\"" Jul 09 22:23:16.912217 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.912204 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 22:23:16.942164 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:16.942121 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:17.003466 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.003440 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5tdn\" (UniqueName: \"kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.003569 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.003526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.005616 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.005585 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:17.062740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.062715 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:23:17.066097 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.066081 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.076740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.076721 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:23:17.104929 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.104854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.104929 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.104923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5tdn\" (UniqueName: \"kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.107222 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.107202 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.117115 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.117095 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5tdn\" (UniqueName: \"kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn\") pod \"maas-controller-b7ccdcc6f-qwlsg\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.151749 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.151707 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:17.206050 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.206027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.206140 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.206071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnp75\" (UniqueName: \"kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.219950 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.219929 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:17.306810 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.306776 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.306976 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.306864 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnp75\" (UniqueName: \"kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.309295 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.309272 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.315965 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.315943 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnp75\" (UniqueName: \"kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75\") pod \"maas-controller-6b7cd97f87-jdb49\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.351175 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.351153 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:17.352618 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:23:17.352597 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986eaa2d_d1b5_4e39_ba35_38326926e799.slice/crio-62a995c1af8fcb81fc8297986b19e725faa9c6ab89bc1a8d4888ff1008c9799b WatchSource:0}: Error finding container 62a995c1af8fcb81fc8297986b19e725faa9c6ab89bc1a8d4888ff1008c9799b: Status 404 returned error can't find the container with id 62a995c1af8fcb81fc8297986b19e725faa9c6ab89bc1a8d4888ff1008c9799b Jul 09 22:23:17.376322 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.376303 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:17.503016 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:17.502991 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:23:17.504647 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:23:17.504620 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf491a46f_7cc7_44bc_a8f8_df9536538ff6.slice/crio-16f5811cc9243ce29a7495b3111867c664af350f682036fd9b3a92372eb9c2d6 WatchSource:0}: Error finding container 16f5811cc9243ce29a7495b3111867c664af350f682036fd9b3a92372eb9c2d6: Status 404 returned error can't find the container with id 16f5811cc9243ce29a7495b3111867c664af350f682036fd9b3a92372eb9c2d6 Jul 09 22:23:18.342977 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:18.342855 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" event={"ID":"f491a46f-7cc7-44bc-a8f8-df9536538ff6","Type":"ContainerStarted","Data":"16f5811cc9243ce29a7495b3111867c664af350f682036fd9b3a92372eb9c2d6"} Jul 09 22:23:18.344832 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:18.344801 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" event={"ID":"986eaa2d-d1b5-4e39-ba35-38326926e799","Type":"ContainerStarted","Data":"62a995c1af8fcb81fc8297986b19e725faa9c6ab89bc1a8d4888ff1008c9799b"} Jul 09 22:23:19.918474 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:19.918439 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:20.019643 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.019520 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:20.356861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.356766 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" event={"ID":"f491a46f-7cc7-44bc-a8f8-df9536538ff6","Type":"ContainerStarted","Data":"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332"} Jul 09 22:23:20.357045 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.356884 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:20.358594 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.358567 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" event={"ID":"986eaa2d-d1b5-4e39-ba35-38326926e799","Type":"ContainerStarted","Data":"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded"} Jul 09 22:23:20.358710 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.358653 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:20.382679 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.382634 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" podStartSLOduration=1.177333276 podStartE2EDuration="3.382622224s" totalImagesPullingTime="2.205288948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:20.380315139 +0000 UTC m=+769.071889193" watchObservedRunningTime="2026-07-09 22:23:20.382622224 +0000 UTC m=+769.074196277" Jul 09 22:23:20.401802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:20.401763 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" podStartSLOduration=2.033846695 podStartE2EDuration="4.401751337s" totalImagesPullingTime="2.367904642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:20.400016699 +0000 UTC m=+769.091590753" watchObservedRunningTime="2026-07-09 22:23:20.401751337 +0000 UTC m=+769.093325390" Jul 09 22:23:31.365999 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.365971 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:31.366567 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.366129 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:23:31.426091 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.426058 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:31.426288 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.426246 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" podUID="986eaa2d-d1b5-4e39-ba35-38326926e799" containerName="manager" containerID="cri-o://8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded" gracePeriod=30 Jul 09 22:23:31.675391 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.675367 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:31.728447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.728423 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert\") pod \"986eaa2d-d1b5-4e39-ba35-38326926e799\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " Jul 09 22:23:31.728581 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.728494 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5tdn\" (UniqueName: \"kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn\") pod \"986eaa2d-d1b5-4e39-ba35-38326926e799\" (UID: \"986eaa2d-d1b5-4e39-ba35-38326926e799\") " Jul 09 22:23:31.730486 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.730456 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "986eaa2d-d1b5-4e39-ba35-38326926e799" (UID: "986eaa2d-d1b5-4e39-ba35-38326926e799"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:23:31.730586 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.730502 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn" (OuterVolumeSpecName: "kube-api-access-g5tdn") pod "986eaa2d-d1b5-4e39-ba35-38326926e799" (UID: "986eaa2d-d1b5-4e39-ba35-38326926e799"). InnerVolumeSpecName "kube-api-access-g5tdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:23:31.829723 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.829691 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/986eaa2d-d1b5-4e39-ba35-38326926e799-webhook-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:23:31.829723 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:31.829720 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5tdn\" (UniqueName: \"kubernetes.io/projected/986eaa2d-d1b5-4e39-ba35-38326926e799-kube-api-access-g5tdn\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:23:32.416641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.416608 2575 generic.go:356] "Generic (PLEG): container finished" podID="986eaa2d-d1b5-4e39-ba35-38326926e799" containerID="8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded" exitCode=0 Jul 09 22:23:32.417033 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.416664 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" Jul 09 22:23:32.417033 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.416694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" event={"ID":"986eaa2d-d1b5-4e39-ba35-38326926e799","Type":"ContainerDied","Data":"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded"} Jul 09 22:23:32.417033 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.416747 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7ccdcc6f-qwlsg" event={"ID":"986eaa2d-d1b5-4e39-ba35-38326926e799","Type":"ContainerDied","Data":"62a995c1af8fcb81fc8297986b19e725faa9c6ab89bc1a8d4888ff1008c9799b"} Jul 09 22:23:32.417033 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.416771 2575 scope.go:117] "RemoveContainer" containerID="8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded" Jul 09 22:23:32.425639 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.425624 2575 scope.go:117] "RemoveContainer" containerID="8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded" Jul 09 22:23:32.425888 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:23:32.425870 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded\": container with ID starting with 8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded not found: ID does not exist" containerID="8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded" Jul 09 22:23:32.425973 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.425914 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded"} err="failed to get container status \"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded\": rpc error: code = NotFound desc = could not find container \"8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded\": container with ID starting with 8add8b6a7e0a20705818bb2abb3c0251dfffbf61c262bb59d2d7ed554195cded not found: ID does not exist" Jul 09 22:23:32.439440 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.439419 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:32.442918 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:32.442880 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b7ccdcc6f-qwlsg"] Jul 09 22:23:33.193520 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:33.193490 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="986eaa2d-d1b5-4e39-ba35-38326926e799" path="/var/lib/kubelet/pods/986eaa2d-d1b5-4e39-ba35-38326926e799/volumes" Jul 09 22:23:44.467824 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:44.467782 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:44.526061 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:44.526035 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:55.508604 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:55.508481 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:55.603044 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:55.603010 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:56.105277 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.105249 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:56.209436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.206784 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-648645db76-nfjcf"] Jul 09 22:23:56.209436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.207649 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="986eaa2d-d1b5-4e39-ba35-38326926e799" containerName="manager" Jul 09 22:23:56.209436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.207670 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="986eaa2d-d1b5-4e39-ba35-38326926e799" containerName="manager" Jul 09 22:23:56.209436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.207875 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="986eaa2d-d1b5-4e39-ba35-38326926e799" containerName="manager" Jul 09 22:23:56.212934 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.212913 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.217461 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.217436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 22:23:56.217601 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.217532 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2qlgm\"" Jul 09 22:23:56.219778 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.219748 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:56.220085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.220065 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-648645db76-nfjcf"] Jul 09 22:23:56.323241 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.323212 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-openshift-service-ca\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.323383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.323251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-maas-api-tls\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.323383 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.323330 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwv7k\" (UniqueName: \"kubernetes.io/projected/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-kube-api-access-zwv7k\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.423719 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.423683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-openshift-service-ca\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.423882 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.423732 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-maas-api-tls\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.423882 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.423772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwv7k\" (UniqueName: \"kubernetes.io/projected/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-kube-api-access-zwv7k\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.424380 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.424357 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-openshift-service-ca\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.426172 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.426150 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-maas-api-tls\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.432630 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.432598 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwv7k\" (UniqueName: \"kubernetes.io/projected/1891e73f-7d7d-4e37-b026-a3c5278d0b1c-kube-api-access-zwv7k\") pod \"maas-api-648645db76-nfjcf\" (UID: \"1891e73f-7d7d-4e37-b026-a3c5278d0b1c\") " pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.524402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.524373 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:56.658598 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.658575 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-648645db76-nfjcf"] Jul 09 22:23:56.660530 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:23:56.660501 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1891e73f_7d7d_4e37_b026_a3c5278d0b1c.slice/crio-015308d4bf488e08e482c0a6dd45f99628d075326a2f937dd1db4c998e3f1e60 WatchSource:0}: Error finding container 015308d4bf488e08e482c0a6dd45f99628d075326a2f937dd1db4c998e3f1e60: Status 404 returned error can't find the container with id 015308d4bf488e08e482c0a6dd45f99628d075326a2f937dd1db4c998e3f1e60 Jul 09 22:23:56.698132 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.698057 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:56.795260 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.795202 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:23:56.814055 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.814032 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k4mxq"] Jul 09 22:23:56.816507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.816492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:56.819329 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.819309 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sqs6v\"" Jul 09 22:23:56.819433 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.819341 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 22:23:56.830589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.830567 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k4mxq"] Jul 09 22:23:56.927831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.927799 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b189cf51-52c4-45b5-bc13-cd7f506e50f6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:56.927831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:56.927833 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7hmg\" (UniqueName: \"kubernetes.io/projected/b189cf51-52c4-45b5-bc13-cd7f506e50f6-kube-api-access-d7hmg\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.029332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.029251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b189cf51-52c4-45b5-bc13-cd7f506e50f6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.029332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.029286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7hmg\" (UniqueName: \"kubernetes.io/projected/b189cf51-52c4-45b5-bc13-cd7f506e50f6-kube-api-access-d7hmg\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.029955 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.029936 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b189cf51-52c4-45b5-bc13-cd7f506e50f6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.038733 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.038710 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7hmg\" (UniqueName: \"kubernetes.io/projected/b189cf51-52c4-45b5-bc13-cd7f506e50f6-kube-api-access-d7hmg\") pod \"payload-pre-processing-7964457696-k4mxq\" (UID: \"b189cf51-52c4-45b5-bc13-cd7f506e50f6\") " pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.126872 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.126834 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:23:57.258198 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.258175 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k4mxq"] Jul 09 22:23:57.260805 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:23:57.260779 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb189cf51_52c4_45b5_bc13_cd7f506e50f6.slice/crio-3b052975ee31daadd304942b5ec6086b5cbd1208ba76a91ef0d871f9928d20f2 WatchSource:0}: Error finding container 3b052975ee31daadd304942b5ec6086b5cbd1208ba76a91ef0d871f9928d20f2: Status 404 returned error can't find the container with id 3b052975ee31daadd304942b5ec6086b5cbd1208ba76a91ef0d871f9928d20f2 Jul 09 22:23:57.537096 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.537045 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" event={"ID":"b189cf51-52c4-45b5-bc13-cd7f506e50f6","Type":"ContainerStarted","Data":"3b052975ee31daadd304942b5ec6086b5cbd1208ba76a91ef0d871f9928d20f2"} Jul 09 22:23:57.538320 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:57.538298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" event={"ID":"1891e73f-7d7d-4e37-b026-a3c5278d0b1c","Type":"ContainerStarted","Data":"015308d4bf488e08e482c0a6dd45f99628d075326a2f937dd1db4c998e3f1e60"} Jul 09 22:23:58.545580 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:58.545539 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" event={"ID":"1891e73f-7d7d-4e37-b026-a3c5278d0b1c","Type":"ContainerStarted","Data":"3d726868533137d16a692d8a03a39e162b77c5a8cc4b63fc43c89c7d6c5222f6"} Jul 09 22:23:58.546053 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:58.545673 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:23:58.569712 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:58.569107 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" podStartSLOduration=0.837675927 podStartE2EDuration="2.569089161s" totalImagesPullingTime="1.731413234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:58.567085729 +0000 UTC m=+807.258659781" watchObservedRunningTime="2026-07-09 22:23:58.569089161 +0000 UTC m=+807.260663215" Jul 09 22:23:59.551781 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:59.551748 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" event={"ID":"b189cf51-52c4-45b5-bc13-cd7f506e50f6","Type":"ContainerStarted","Data":"b4ce40bb643ac624f6db29c30842318f7606a3c195414ee357b8b9903d3bdfa3"} Jul 09 22:23:59.571763 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:23:59.571686 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" podStartSLOduration=2.066666344 podStartE2EDuration="3.571672145s" totalImagesPullingTime="1.505005801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:59.570949904 +0000 UTC m=+808.262523954" watchObservedRunningTime="2026-07-09 22:23:59.571672145 +0000 UTC m=+808.263246198" Jul 09 22:24:00.556923 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:00.556883 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:24:04.557570 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:04.557524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-648645db76-nfjcf" Jul 09 22:24:05.563714 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:05.563677 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:05.650114 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:05.650090 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:11.562875 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:11.562843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-k4mxq" Jul 09 22:24:21.194190 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.194158 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4"] Jul 09 22:24:21.199436 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.199413 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.203713 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.203680 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 22:24:21.203843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.203721 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jn279\"" Jul 09 22:24:21.203843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.203750 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 22:24:21.203843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.203720 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 22:24:21.210457 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.210437 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4"] Jul 09 22:24:21.228811 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.228785 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:21.337629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337592 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.337757 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.337757 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.337832 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.337832 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.337832 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.337828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc2lh\" (UniqueName: \"kubernetes.io/projected/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-kube-api-access-dc2lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.438725 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438700 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.438873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.438873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc2lh\" (UniqueName: \"kubernetes.io/projected/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-kube-api-access-dc2lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.438873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.439069 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.439069 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.438926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.439190 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.439166 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.439254 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.439223 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.439313 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.439289 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.440982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.440961 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.441310 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.441293 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.447294 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.447243 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc2lh\" (UniqueName: \"kubernetes.io/projected/6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7-kube-api-access-dc2lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4\" (UID: \"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.511129 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.511105 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:21.645856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.645830 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4"] Jul 09 22:24:21.647647 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:24:21.647614 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb620bb_31dc_41a5_9e4e_fb21d7e9b4d7.slice/crio-1751f73a49d56b8b3c588b9a63998e99f4334b45a32cdc3a7bfc0fc741ee2ff4 WatchSource:0}: Error finding container 1751f73a49d56b8b3c588b9a63998e99f4334b45a32cdc3a7bfc0fc741ee2ff4: Status 404 returned error can't find the container with id 1751f73a49d56b8b3c588b9a63998e99f4334b45a32cdc3a7bfc0fc741ee2ff4 Jul 09 22:24:21.664954 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:21.664924 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" event={"ID":"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7","Type":"ContainerStarted","Data":"1751f73a49d56b8b3c588b9a63998e99f4334b45a32cdc3a7bfc0fc741ee2ff4"} Jul 09 22:24:23.676627 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:23.676587 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" event={"ID":"6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7","Type":"ContainerStarted","Data":"0f2ebc6530d2b1377088b66734f91ac3738fd26aa07bf95af48e83cd4565a46c"} Jul 09 22:24:23.695992 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:23.695940 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" podStartSLOduration=1.638007135 podStartE2EDuration="2.695922722s" totalImagesPullingTime="1.057915587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:24:23.695303372 +0000 UTC m=+832.386877424" watchObservedRunningTime="2026-07-09 22:24:23.695922722 +0000 UTC m=+832.387496776" Jul 09 22:24:30.947494 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:30.947461 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:31.511887 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:31.511860 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:31.512067 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:31.511935 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:31.524365 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:31.524344 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:31.726117 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:31.726091 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4" Jul 09 22:24:44.066826 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:44.066790 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:47.206367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.206340 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh"] Jul 09 22:24:47.209750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.209723 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.214514 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.214490 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 22:24:47.220410 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.220387 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh"] Jul 09 22:24:47.238009 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.237978 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:47.254491 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254461 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.254645 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254619 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4027d33b-a2f9-485d-8e43-b20d433bd18e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.254715 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.254768 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254715 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.254837 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.254990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.254853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75vlx\" (UniqueName: \"kubernetes.io/projected/4027d33b-a2f9-485d-8e43-b20d433bd18e-kube-api-access-75vlx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.355765 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.355958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4027d33b-a2f9-485d-8e43-b20d433bd18e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.355958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355860 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.355958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.355958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.356187 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.355980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vlx\" (UniqueName: \"kubernetes.io/projected/4027d33b-a2f9-485d-8e43-b20d433bd18e-kube-api-access-75vlx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.356415 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.356379 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.356504 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.356380 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.356504 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.356380 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.358482 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.358460 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4027d33b-a2f9-485d-8e43-b20d433bd18e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.358845 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.358822 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4027d33b-a2f9-485d-8e43-b20d433bd18e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.364683 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.364664 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75vlx\" (UniqueName: \"kubernetes.io/projected/4027d33b-a2f9-485d-8e43-b20d433bd18e-kube-api-access-75vlx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-86dlh\" (UID: \"4027d33b-a2f9-485d-8e43-b20d433bd18e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.521581 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.521494 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:47.654648 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.654623 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh"] Jul 09 22:24:47.656950 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:24:47.656921 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4027d33b_a2f9_485d_8e43_b20d433bd18e.slice/crio-fe9b42cb22499f6fc64d687a81d5d4188441a78436449f381b987b7af1f5e7c5 WatchSource:0}: Error finding container fe9b42cb22499f6fc64d687a81d5d4188441a78436449f381b987b7af1f5e7c5: Status 404 returned error can't find the container with id fe9b42cb22499f6fc64d687a81d5d4188441a78436449f381b987b7af1f5e7c5 Jul 09 22:24:47.792175 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:47.792090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" event={"ID":"4027d33b-a2f9-485d-8e43-b20d433bd18e","Type":"ContainerStarted","Data":"fe9b42cb22499f6fc64d687a81d5d4188441a78436449f381b987b7af1f5e7c5"} Jul 09 22:24:48.799546 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:48.799510 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" event={"ID":"4027d33b-a2f9-485d-8e43-b20d433bd18e","Type":"ContainerStarted","Data":"bc5f20d53e3006126bb14fcddec4d6cb13165da316e1dd3421b45ec58bc26830"} Jul 09 22:24:48.819942 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:48.819872 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" podStartSLOduration=1.654841158 podStartE2EDuration="1.819857999s" totalImagesPullingTime="165.016841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:24:48.819019495 +0000 UTC m=+857.510593553" watchObservedRunningTime="2026-07-09 22:24:48.819857999 +0000 UTC m=+857.511432055" Jul 09 22:24:50.495323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.495290 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r"] Jul 09 22:24:50.498011 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.497986 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.501072 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.501041 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 22:24:50.512107 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.512085 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r"] Jul 09 22:24:50.562429 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.562398 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:24:50.586584 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586558 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.586708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586586 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.586708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.586708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586659 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.586708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.586862 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.586721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nhct\" (UniqueName: \"kubernetes.io/projected/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-kube-api-access-2nhct\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.687804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.687668 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nhct\" (UniqueName: \"kubernetes.io/projected/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-kube-api-access-2nhct\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.688149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.688125 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.688219 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.688147 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.688219 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.688187 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.690281 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.690260 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.690447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.690431 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.697256 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.697231 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nhct\" (UniqueName: \"kubernetes.io/projected/db7f7cdc-7167-4cff-8a08-dcd8d96b52b6-kube-api-access-2nhct\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r\" (UID: \"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.808773 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.808689 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:24:50.940785 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:50.940764 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r"] Jul 09 22:24:50.942580 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:24:50.942551 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7f7cdc_7167_4cff_8a08_dcd8d96b52b6.slice/crio-f9cb9a667db8535d4615b2e227d836b82584ce2d2d38642a331507c659d3c539 WatchSource:0}: Error finding container f9cb9a667db8535d4615b2e227d836b82584ce2d2d38642a331507c659d3c539: Status 404 returned error can't find the container with id f9cb9a667db8535d4615b2e227d836b82584ce2d2d38642a331507c659d3c539 Jul 09 22:24:51.816227 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:51.816196 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" event={"ID":"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6","Type":"ContainerStarted","Data":"79afcab7d9d792a1597a81d8341c2b6ed0a40cae063023008c3344c40f1216d0"} Jul 09 22:24:51.816620 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:51.816249 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" event={"ID":"db7f7cdc-7167-4cff-8a08-dcd8d96b52b6","Type":"ContainerStarted","Data":"f9cb9a667db8535d4615b2e227d836b82584ce2d2d38642a331507c659d3c539"} Jul 09 22:24:51.836283 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:51.836240 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" podStartSLOduration=1.6533681580000001 podStartE2EDuration="1.836226495s" totalImagesPullingTime="182.858337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:24:51.835023259 +0000 UTC m=+860.526597328" watchObservedRunningTime="2026-07-09 22:24:51.836226495 +0000 UTC m=+860.527800548" Jul 09 22:24:57.521958 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:57.521930 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:57.522421 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:57.521984 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:57.534500 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:57.534475 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:24:57.855422 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:24:57.855351 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-86dlh" Jul 09 22:25:00.808910 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:00.808856 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:25:00.809291 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:00.808963 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:25:00.821632 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:00.821607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:25:00.871610 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:00.871586 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r" Jul 09 22:25:12.904970 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:12.904936 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c"] Jul 09 22:25:12.908063 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:12.908039 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:12.910950 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:12.910928 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 22:25:12.920674 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:12.920650 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c"] Jul 09 22:25:12.934142 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:12.934116 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:25:13.081701 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081663 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.081957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84glb\" (UniqueName: \"kubernetes.io/projected/33892e01-8822-4dac-8037-6cd49cc51ee5-kube-api-access-84glb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.081957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081737 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.081957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.081957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081875 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33892e01-8822-4dac-8037-6cd49cc51ee5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.081957 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.081927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182687 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182687 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33892e01-8822-4dac-8037-6cd49cc51ee5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182687 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182750 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84glb\" (UniqueName: \"kubernetes.io/projected/33892e01-8822-4dac-8037-6cd49cc51ee5-kube-api-access-84glb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.182968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.182797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.183171 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.183149 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.183210 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.183192 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.183210 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.183202 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.184915 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.184882 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33892e01-8822-4dac-8037-6cd49cc51ee5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.185182 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.185166 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33892e01-8822-4dac-8037-6cd49cc51ee5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.193348 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.193324 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84glb\" (UniqueName: \"kubernetes.io/projected/33892e01-8822-4dac-8037-6cd49cc51ee5-kube-api-access-84glb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8sr8c\" (UID: \"33892e01-8822-4dac-8037-6cd49cc51ee5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.220513 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.220495 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:13.351769 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.351744 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c"] Jul 09 22:25:13.353881 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:25:13.353855 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33892e01_8822_4dac_8037_6cd49cc51ee5.slice/crio-8ed4f5b3a76646c57a6909dfe93661637fb213512fc636083e1e3dd9126d7259 WatchSource:0}: Error finding container 8ed4f5b3a76646c57a6909dfe93661637fb213512fc636083e1e3dd9126d7259: Status 404 returned error can't find the container with id 8ed4f5b3a76646c57a6909dfe93661637fb213512fc636083e1e3dd9126d7259 Jul 09 22:25:13.928560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.928529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" event={"ID":"33892e01-8822-4dac-8037-6cd49cc51ee5","Type":"ContainerStarted","Data":"b11b0711c3973a589aeb3c068c311d5a65c2f235f169818ce4abec70e9fd4288"} Jul 09 22:25:13.928956 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.928577 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" event={"ID":"33892e01-8822-4dac-8037-6cd49cc51ee5","Type":"ContainerStarted","Data":"8ed4f5b3a76646c57a6909dfe93661637fb213512fc636083e1e3dd9126d7259"} Jul 09 22:25:13.947866 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:13.947814 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" podStartSLOduration=1.775039102 podStartE2EDuration="1.947795694s" totalImagesPullingTime="172.756592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:25:13.947367543 +0000 UTC m=+882.638941597" watchObservedRunningTime="2026-07-09 22:25:13.947795694 +0000 UTC m=+882.639369748" Jul 09 22:25:23.220758 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:23.220722 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:23.221173 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:23.220788 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:23.233237 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:23.233210 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:23.732417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:23.732392 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:25:23.990071 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:23.989997 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8sr8c" Jul 09 22:25:32.377980 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.377955 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:25:32.385979 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.385955 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:25:32.386132 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.386022 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:25:32.387661 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.387636 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:25:32.390857 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.390841 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:25:32.392691 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:25:32.392675 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:26:25.717332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:26:25.717293 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:26:25.929720 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:26:25.929680 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:26:26.866818 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:26:26.866789 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:26:26.966999 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:26:26.966965 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:26:42.339513 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:26:42.339471 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:27:26.840830 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:26.840750 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:27:37.809919 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:37.809856 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:27:37.914735 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:37.914702 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:27:37.914946 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:37.914922 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" podUID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" containerName="manager" containerID="cri-o://5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332" gracePeriod=30 Jul 09 22:27:38.166747 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.166718 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:27:38.210345 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.210314 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnp75\" (UniqueName: \"kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75\") pod \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " Jul 09 22:27:38.210503 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.210491 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert\") pod \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\" (UID: \"f491a46f-7cc7-44bc-a8f8-df9536538ff6\") " Jul 09 22:27:38.212389 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.212356 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75" (OuterVolumeSpecName: "kube-api-access-pnp75") pod "f491a46f-7cc7-44bc-a8f8-df9536538ff6" (UID: "f491a46f-7cc7-44bc-a8f8-df9536538ff6"). InnerVolumeSpecName "kube-api-access-pnp75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:27:38.212762 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.212740 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f491a46f-7cc7-44bc-a8f8-df9536538ff6" (UID: "f491a46f-7cc7-44bc-a8f8-df9536538ff6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:27:38.311794 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.311769 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f491a46f-7cc7-44bc-a8f8-df9536538ff6-webhook-cert\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:27:38.311794 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.311790 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnp75\" (UniqueName: \"kubernetes.io/projected/f491a46f-7cc7-44bc-a8f8-df9536538ff6-kube-api-access-pnp75\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:27:38.650915 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.650856 2575 generic.go:356] "Generic (PLEG): container finished" podID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" containerID="5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332" exitCode=0 Jul 09 22:27:38.650915 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.650915 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" Jul 09 22:27:38.651139 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.650932 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" event={"ID":"f491a46f-7cc7-44bc-a8f8-df9536538ff6","Type":"ContainerDied","Data":"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332"} Jul 09 22:27:38.651139 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.650994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-jdb49" event={"ID":"f491a46f-7cc7-44bc-a8f8-df9536538ff6","Type":"ContainerDied","Data":"16f5811cc9243ce29a7495b3111867c664af350f682036fd9b3a92372eb9c2d6"} Jul 09 22:27:38.651139 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.651024 2575 scope.go:117] "RemoveContainer" containerID="5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332" Jul 09 22:27:38.660123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.660111 2575 scope.go:117] "RemoveContainer" containerID="5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332" Jul 09 22:27:38.660358 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:27:38.660342 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332\": container with ID starting with 5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332 not found: ID does not exist" containerID="5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332" Jul 09 22:27:38.660411 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.660365 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332"} err="failed to get container status \"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332\": rpc error: code = NotFound desc = could not find container \"5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332\": container with ID starting with 5a8da4906dcdad9535679f1100f8ea3d213eab320f49d23dc80cb20c4d44d332 not found: ID does not exist" Jul 09 22:27:38.673591 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.673541 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:27:38.675608 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:38.675588 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-jdb49"] Jul 09 22:27:39.192861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.192826 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" path="/var/lib/kubelet/pods/f491a46f-7cc7-44bc-a8f8-df9536538ff6/volumes" Jul 09 22:27:39.329105 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.329075 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:27:39.434559 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.434527 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-gjkbm"] Jul 09 22:27:39.435558 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.435513 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" containerName="manager" Jul 09 22:27:39.435558 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.435558 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" containerName="manager" Jul 09 22:27:39.435905 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.435876 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f491a46f-7cc7-44bc-a8f8-df9536538ff6" containerName="manager" Jul 09 22:27:39.441125 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.441100 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.444437 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.444235 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 22:27:39.445635 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.445614 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dpv7x\"" Jul 09 22:27:39.445755 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.445624 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 22:27:39.446814 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.446794 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-gjkbm"] Jul 09 22:27:39.521259 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.521232 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-webhook-cert\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.521371 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.521275 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9fv\" (UniqueName: \"kubernetes.io/projected/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-kube-api-access-2l9fv\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.622719 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.622689 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-webhook-cert\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.622889 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.622729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9fv\" (UniqueName: \"kubernetes.io/projected/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-kube-api-access-2l9fv\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.625073 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.625049 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-webhook-cert\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.631856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.631830 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9fv\" (UniqueName: \"kubernetes.io/projected/cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4-kube-api-access-2l9fv\") pod \"maas-controller-6b7cd97f87-gjkbm\" (UID: \"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4\") " pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.753823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.753751 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:39.883594 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.883567 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7cd97f87-gjkbm"] Jul 09 22:27:39.885337 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:27:39.885304 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacd91b7_8d35_429d_aab5_b7d6ec3ddfd4.slice/crio-db58cefbc4bfb35354aa4d8c5565205215f39fc49083470f18b56d5a0c1e7ba9 WatchSource:0}: Error finding container db58cefbc4bfb35354aa4d8c5565205215f39fc49083470f18b56d5a0c1e7ba9: Status 404 returned error can't find the container with id db58cefbc4bfb35354aa4d8c5565205215f39fc49083470f18b56d5a0c1e7ba9 Jul 09 22:27:39.886747 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:39.886731 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:27:40.664878 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:40.664847 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" event={"ID":"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4","Type":"ContainerStarted","Data":"47a751f47d180b05d2773e8e7b2ae87b693b0edaf08d426501163945c035d4f8"} Jul 09 22:27:40.665249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:40.664916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" event={"ID":"cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4","Type":"ContainerStarted","Data":"db58cefbc4bfb35354aa4d8c5565205215f39fc49083470f18b56d5a0c1e7ba9"} Jul 09 22:27:40.665249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:40.664938 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:40.683332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:40.683290 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" podStartSLOduration=1.189099328 podStartE2EDuration="1.683275457s" totalImagesPullingTime="494.176129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:27:40.682260582 +0000 UTC m=+1029.373834659" watchObservedRunningTime="2026-07-09 22:27:40.683275457 +0000 UTC m=+1029.374849509" Jul 09 22:27:51.672815 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:51.672783 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b7cd97f87-gjkbm" Jul 09 22:27:52.722173 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:27:52.722140 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:03.208506 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:03.208479 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:23.116929 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:23.116878 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:23.233454 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:23.233425 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:23.339764 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:23.339689 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:23.736570 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:23.736541 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:29.316296 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:29.316263 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:29.415143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:29.415114 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:29.519368 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:29.519331 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:29.649904 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:29.649873 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:35.496350 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:35.496301 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:28:35.603332 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:28:35.603299 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:30:05.937823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:05.937736 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:30:22.842567 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:22.842529 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:30:22.954330 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:22.954300 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:30:32.414185 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.414156 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:30:32.415994 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.415966 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:30:32.418936 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.418917 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:30:32.420570 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.420550 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:30:32.420732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.420713 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:30:32.425253 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:30:32.425239 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:32:28.863714 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:28.863680 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:32:28.865997 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:28.863868 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" podUID="683e5009-8fa4-4979-9d95-bec3d6264354" containerName="manager" containerID="cri-o://e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b" gracePeriod=10 Jul 09 22:32:32.515866 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.515844 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:32:32.630802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.630733 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume\") pod \"683e5009-8fa4-4979-9d95-bec3d6264354\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " Jul 09 22:32:32.630802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.630793 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djmdj\" (UniqueName: \"kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj\") pod \"683e5009-8fa4-4979-9d95-bec3d6264354\" (UID: \"683e5009-8fa4-4979-9d95-bec3d6264354\") " Jul 09 22:32:32.631083 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.631061 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "683e5009-8fa4-4979-9d95-bec3d6264354" (UID: "683e5009-8fa4-4979-9d95-bec3d6264354"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:32:32.632666 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.632648 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj" (OuterVolumeSpecName: "kube-api-access-djmdj") pod "683e5009-8fa4-4979-9d95-bec3d6264354" (UID: "683e5009-8fa4-4979-9d95-bec3d6264354"). InnerVolumeSpecName "kube-api-access-djmdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:32:32.732370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.732333 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djmdj\" (UniqueName: \"kubernetes.io/projected/683e5009-8fa4-4979-9d95-bec3d6264354-kube-api-access-djmdj\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:32:32.732370 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:32.732368 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/683e5009-8fa4-4979-9d95-bec3d6264354-extensions-socket-volume\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:32:33.151987 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.151951 2575 generic.go:356] "Generic (PLEG): container finished" podID="683e5009-8fa4-4979-9d95-bec3d6264354" containerID="e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b" exitCode=0 Jul 09 22:32:33.152169 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.152003 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" Jul 09 22:32:33.152169 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.152027 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" event={"ID":"683e5009-8fa4-4979-9d95-bec3d6264354","Type":"ContainerDied","Data":"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b"} Jul 09 22:32:33.152169 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.152081 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6" event={"ID":"683e5009-8fa4-4979-9d95-bec3d6264354","Type":"ContainerDied","Data":"4c3d41324db1e91ecfe5fdf74e84cacbb2521c04a02baa2f90b12004af187f93"} Jul 09 22:32:33.152169 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.152104 2575 scope.go:117] "RemoveContainer" containerID="e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b" Jul 09 22:32:33.161738 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.161725 2575 scope.go:117] "RemoveContainer" containerID="e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b" Jul 09 22:32:33.161984 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:32:33.161968 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b\": container with ID starting with e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b not found: ID does not exist" containerID="e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b" Jul 09 22:32:33.162042 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.161992 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b"} err="failed to get container status \"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b\": rpc error: code = NotFound desc = could not find container \"e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b\": container with ID starting with e4c9b2064bab18a6986ea7c6c7269c3a2e178322b34212e78eb36a0c7af7f70b not found: ID does not exist" Jul 09 22:32:33.176097 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.176074 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:32:33.179830 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.179810 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wlrj6"] Jul 09 22:32:33.198992 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:32:33.198970 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683e5009-8fa4-4979-9d95-bec3d6264354" path="/var/lib/kubelet/pods/683e5009-8fa4-4979-9d95-bec3d6264354/volumes" Jul 09 22:33:29.998888 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:29.998812 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt"] Jul 09 22:33:29.999346 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:29.999255 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683e5009-8fa4-4979-9d95-bec3d6264354" containerName="manager" Jul 09 22:33:29.999346 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:29.999270 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="683e5009-8fa4-4979-9d95-bec3d6264354" containerName="manager" Jul 09 22:33:29.999346 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:29.999342 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="683e5009-8fa4-4979-9d95-bec3d6264354" containerName="manager" Jul 09 22:33:30.002433 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.002414 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.005427 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.005408 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ngbqm\"" Jul 09 22:33:30.018780 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.018759 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt"] Jul 09 22:33:30.159051 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.159019 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzh9\" (UniqueName: \"kubernetes.io/projected/c092e5c8-645a-40e7-9049-2246a4aa3d83-kube-api-access-8rzh9\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.159198 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.159153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c092e5c8-645a-40e7-9049-2246a4aa3d83-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.260381 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.260288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c092e5c8-645a-40e7-9049-2246a4aa3d83-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.260381 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.260351 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzh9\" (UniqueName: \"kubernetes.io/projected/c092e5c8-645a-40e7-9049-2246a4aa3d83-kube-api-access-8rzh9\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.260650 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.260629 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c092e5c8-645a-40e7-9049-2246a4aa3d83-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.269301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.269277 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzh9\" (UniqueName: \"kubernetes.io/projected/c092e5c8-645a-40e7-9049-2246a4aa3d83-kube-api-access-8rzh9\") pod \"kuadrant-operator-controller-manager-86878c769f-jp6tt\" (UID: \"c092e5c8-645a-40e7-9049-2246a4aa3d83\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.312802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.312783 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:30.457188 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:30.457100 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt"] Jul 09 22:33:30.459705 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:33:30.459666 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc092e5c8_645a_40e7_9049_2246a4aa3d83.slice/crio-1b83bb1433c6e134af11ab583a49ae2d78e4e666d43e32a438c55e4e6168a731 WatchSource:0}: Error finding container 1b83bb1433c6e134af11ab583a49ae2d78e4e666d43e32a438c55e4e6168a731: Status 404 returned error can't find the container with id 1b83bb1433c6e134af11ab583a49ae2d78e4e666d43e32a438c55e4e6168a731 Jul 09 22:33:31.430150 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:31.430112 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" event={"ID":"c092e5c8-645a-40e7-9049-2246a4aa3d83","Type":"ContainerStarted","Data":"2cbd3aac10d08cfda4e7c96529d751d8d63d0e973c0e21c90fad065359dc8d86"} Jul 09 22:33:31.430560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:31.430167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" event={"ID":"c092e5c8-645a-40e7-9049-2246a4aa3d83","Type":"ContainerStarted","Data":"1b83bb1433c6e134af11ab583a49ae2d78e4e666d43e32a438c55e4e6168a731"} Jul 09 22:33:31.430560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:31.430190 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:33:31.455269 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:31.455220 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" podStartSLOduration=2.455206423 podStartE2EDuration="2.455206423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:33:31.453817209 +0000 UTC m=+1380.145391277" watchObservedRunningTime="2026-07-09 22:33:31.455206423 +0000 UTC m=+1380.146780476" Jul 09 22:33:42.437229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:33:42.437202 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jp6tt" Jul 09 22:34:30.972493 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:34:30.972418 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:34:31.060319 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:34:31.060265 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:35:19.992843 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:19.992803 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:35:21.711938 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:21.711875 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:35:32.447732 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.447703 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:35:32.449746 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.449728 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:35:32.455215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.455198 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:35:32.455316 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.455252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:35:32.457143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.457122 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:35:32.462191 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:35:32.462177 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:36:26.941931 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:36:26.941833 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:24.537020 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:24.536929 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:29.834318 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:29.834277 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:38.091326 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:38.091288 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:43.191218 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:43.191181 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:46.002089 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:46.002056 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:46.178793 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:46.178753 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:46.484717 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:46.484681 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:52.447461 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:52.447428 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:52.513178 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:52.513139 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:52.700123 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:52.700047 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:52.799786 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:52.799751 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:53.900158 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:53.900125 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:53.973853 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:53.973821 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:39:53.977685 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:53.977664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:53.981360 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:53.981190 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-489a4fc6-serving-cert\"" Jul 09 22:39:53.995085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:53.995066 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:39:54.007840 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.007808 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:54.029801 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.029775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.029918 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.029840 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.029971 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.029952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq8b7\" (UniqueName: \"kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.114553 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.114519 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:54.130676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.130648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.130802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.130697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.130802 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.130742 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq8b7\" (UniqueName: \"kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.130937 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:39:54.130806 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-serving-cert: secret "maas-api-e2e-ait-489a4fc6-serving-cert" not found Jul 09 22:39:54.130937 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:39:54.130867 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls podName:32874a77-80bc-4a0c-a1d9-02da1f728ad7 nodeName:}" failed. No retries permitted until 2026-07-09 22:39:54.630847535 +0000 UTC m=+1763.322421581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls") pod "maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" (UID: "32874a77-80bc-4a0c-a1d9-02da1f728ad7") : secret "maas-api-e2e-ait-489a4fc6-serving-cert" not found Jul 09 22:39:54.131323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.131302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.144630 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.144606 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq8b7\" (UniqueName: \"kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.241934 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.241832 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:54.571845 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.571753 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:54.635805 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.635777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.638163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.638139 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") pod \"maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:54.667775 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.667749 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:39:54.890069 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:54.889990 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:55.030553 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:55.030520 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:39:55.030963 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:39:55.030811 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32874a77_80bc_4a0c_a1d9_02da1f728ad7.slice/crio-e820b4ab28b7cdff7f197fb93b861c938b061673067472dc80d796c6af90f0a6 WatchSource:0}: Error finding container e820b4ab28b7cdff7f197fb93b861c938b061673067472dc80d796c6af90f0a6: Status 404 returned error can't find the container with id e820b4ab28b7cdff7f197fb93b861c938b061673067472dc80d796c6af90f0a6 Jul 09 22:39:55.032388 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:55.032366 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:39:55.346647 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:55.346614 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" event={"ID":"32874a77-80bc-4a0c-a1d9-02da1f728ad7","Type":"ContainerStarted","Data":"e820b4ab28b7cdff7f197fb93b861c938b061673067472dc80d796c6af90f0a6"} Jul 09 22:39:56.353568 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:56.353535 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" event={"ID":"32874a77-80bc-4a0c-a1d9-02da1f728ad7","Type":"ContainerStarted","Data":"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb"} Jul 09 22:39:56.353993 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:56.353600 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:39:56.371529 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:56.371471 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" podStartSLOduration=2.934811057 podStartE2EDuration="3.371450526s" totalImagesPullingTime="436.639469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:39:56.370971317 +0000 UTC m=+1765.062545393" watchObservedRunningTime="2026-07-09 22:39:56.371450526 +0000 UTC m=+1765.063024584" Jul 09 22:39:56.454015 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:39:56.453967 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:40:02.364665 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:02.364635 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:40:32.486389 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.486354 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:40:32.488138 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.488114 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:40:32.492622 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.492603 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:40:32.492825 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.492807 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:40:32.494680 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.494660 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:40:32.498940 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:32.498922 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:40:53.893378 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:53.893345 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:40:55.082416 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:55.082374 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:40:58.998162 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:58.998117 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:40:59.114150 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:40:59.114118 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:05.016127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.016095 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:41:05.016623 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.016281 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" podUID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" containerName="maas-api" containerID="cri-o://49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb" gracePeriod=30 Jul 09 22:41:05.049349 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.049322 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:05.130451 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.130424 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:05.273294 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.273231 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:41:05.380287 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.380254 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") pod \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " Jul 09 22:41:05.380476 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.380395 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq8b7\" (UniqueName: \"kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7\") pod \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " Jul 09 22:41:05.380476 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.380431 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca\") pod \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\" (UID: \"32874a77-80bc-4a0c-a1d9-02da1f728ad7\") " Jul 09 22:41:05.380837 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.380813 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32874a77-80bc-4a0c-a1d9-02da1f728ad7" (UID: "32874a77-80bc-4a0c-a1d9-02da1f728ad7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:41:05.382355 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.382334 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7" (OuterVolumeSpecName: "kube-api-access-zq8b7") pod "32874a77-80bc-4a0c-a1d9-02da1f728ad7" (UID: "32874a77-80bc-4a0c-a1d9-02da1f728ad7"). InnerVolumeSpecName "kube-api-access-zq8b7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:41:05.382415 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.382364 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32874a77-80bc-4a0c-a1d9-02da1f728ad7" (UID: "32874a77-80bc-4a0c-a1d9-02da1f728ad7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:41:05.454744 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.454711 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:05.481513 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.481485 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq8b7\" (UniqueName: \"kubernetes.io/projected/32874a77-80bc-4a0c-a1d9-02da1f728ad7-kube-api-access-zq8b7\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:05.481513 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.481509 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32874a77-80bc-4a0c-a1d9-02da1f728ad7-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:05.481653 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.481523 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32874a77-80bc-4a0c-a1d9-02da1f728ad7-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:05.694848 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.694819 2575 generic.go:356] "Generic (PLEG): container finished" podID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" containerID="49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb" exitCode=0 Jul 09 22:41:05.695043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.694861 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" Jul 09 22:41:05.695043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.694870 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" event={"ID":"32874a77-80bc-4a0c-a1d9-02da1f728ad7","Type":"ContainerDied","Data":"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb"} Jul 09 22:41:05.695043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.694944 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw" event={"ID":"32874a77-80bc-4a0c-a1d9-02da1f728ad7","Type":"ContainerDied","Data":"e820b4ab28b7cdff7f197fb93b861c938b061673067472dc80d796c6af90f0a6"} Jul 09 22:41:05.695043 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.694967 2575 scope.go:117] "RemoveContainer" containerID="49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb" Jul 09 22:41:05.703561 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.703530 2575 scope.go:117] "RemoveContainer" containerID="49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb" Jul 09 22:41:05.703794 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:41:05.703776 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb\": container with ID starting with 49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb not found: ID does not exist" containerID="49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb" Jul 09 22:41:05.703840 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.703803 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb"} err="failed to get container status \"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb\": rpc error: code = NotFound desc = could not find container \"49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb\": container with ID starting with 49b2a726c6f02c2c8d08e4966cc7a07c0534646fdc254050868ade4b4a2999fb not found: ID does not exist" Jul 09 22:41:05.718823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.718801 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:41:05.723766 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:05.723744 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-489a4fc6-5fd6b469d6-sdlxw"] Jul 09 22:41:07.192589 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:07.192555 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" path="/var/lib/kubelet/pods/32874a77-80bc-4a0c-a1d9-02da1f728ad7/volumes" Jul 09 22:41:10.776913 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:10.776860 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:10.880101 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:10.880069 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:11.013574 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:11.013545 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:11.111861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:11.111763 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:12.193417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.193384 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:12.292824 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.292786 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:12.412964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.412932 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:12.413544 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.413525 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" containerName="maas-api" Jul 09 22:41:12.413616 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.413548 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" containerName="maas-api" Jul 09 22:41:12.413700 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.413688 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="32874a77-80bc-4a0c-a1d9-02da1f728ad7" containerName="maas-api" Jul 09 22:41:12.418532 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.418510 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.421435 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.421411 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fdfdaecd-serving-cert\"" Jul 09 22:41:12.431695 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.431673 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:12.453431 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.453360 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:12.545754 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.545721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.545930 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.545780 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.545930 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.545858 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jv9t\" (UniqueName: \"kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.601203 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.601178 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:12.646853 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.646828 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.646974 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.646878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.646974 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.646934 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jv9t\" (UniqueName: \"kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.647472 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.647452 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.649209 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.649190 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.655730 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.655702 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jv9t\" (UniqueName: \"kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t\") pod \"maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.732431 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.732374 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:12.867386 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:12.867356 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:12.869557 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:41:12.869532 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b54299_a944_4225_81d7_e637941585d8.slice/crio-638ba5c21f9809560ee678936a50dff7414f3f8c9d0b9ab6e1c66de08da3fdd0 WatchSource:0}: Error finding container 638ba5c21f9809560ee678936a50dff7414f3f8c9d0b9ab6e1c66de08da3fdd0: Status 404 returned error can't find the container with id 638ba5c21f9809560ee678936a50dff7414f3f8c9d0b9ab6e1c66de08da3fdd0 Jul 09 22:41:13.013259 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.013170 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:13.113772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.113735 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:13.742498 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.742461 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" event={"ID":"d0b54299-a944-4225-81d7-e637941585d8","Type":"ContainerStarted","Data":"8e118310f008482424fbaa3b3d3609176d26d5bce4a09b19398f2c1a45e8d9ec"} Jul 09 22:41:13.742876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.742523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" event={"ID":"d0b54299-a944-4225-81d7-e637941585d8","Type":"ContainerStarted","Data":"638ba5c21f9809560ee678936a50dff7414f3f8c9d0b9ab6e1c66de08da3fdd0"} Jul 09 22:41:13.742876 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.742639 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:13.771548 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:13.771494 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" podStartSLOduration=1.387990733 podStartE2EDuration="1.771473479s" totalImagesPullingTime="383.482746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:41:13.770555463 +0000 UTC m=+1842.462129518" watchObservedRunningTime="2026-07-09 22:41:13.771473479 +0000 UTC m=+1842.463047533" Jul 09 22:41:14.272873 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:14.272841 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:19.753265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:19.753239 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:27.648786 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.648750 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:27.649277 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.649065 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" podUID="d0b54299-a944-4225-81d7-e637941585d8" containerName="maas-api" containerID="cri-o://8e118310f008482424fbaa3b3d3609176d26d5bce4a09b19398f2c1a45e8d9ec" gracePeriod=30 Jul 09 22:41:27.679320 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.679292 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:27.817772 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.817740 2575 generic.go:356] "Generic (PLEG): container finished" podID="d0b54299-a944-4225-81d7-e637941585d8" containerID="8e118310f008482424fbaa3b3d3609176d26d5bce4a09b19398f2c1a45e8d9ec" exitCode=0 Jul 09 22:41:27.817993 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.817812 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" event={"ID":"d0b54299-a944-4225-81d7-e637941585d8","Type":"ContainerDied","Data":"8e118310f008482424fbaa3b3d3609176d26d5bce4a09b19398f2c1a45e8d9ec"} Jul 09 22:41:27.906528 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.906470 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:27.987444 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.987411 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca\") pod \"d0b54299-a944-4225-81d7-e637941585d8\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " Jul 09 22:41:27.987622 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.987504 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jv9t\" (UniqueName: \"kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t\") pod \"d0b54299-a944-4225-81d7-e637941585d8\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " Jul 09 22:41:27.987622 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.987556 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls\") pod \"d0b54299-a944-4225-81d7-e637941585d8\" (UID: \"d0b54299-a944-4225-81d7-e637941585d8\") " Jul 09 22:41:27.987844 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.987809 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0b54299-a944-4225-81d7-e637941585d8" (UID: "d0b54299-a944-4225-81d7-e637941585d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:41:27.989470 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.989446 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t" (OuterVolumeSpecName: "kube-api-access-2jv9t") pod "d0b54299-a944-4225-81d7-e637941585d8" (UID: "d0b54299-a944-4225-81d7-e637941585d8"). InnerVolumeSpecName "kube-api-access-2jv9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:41:27.989581 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:27.989537 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0b54299-a944-4225-81d7-e637941585d8" (UID: "d0b54299-a944-4225-81d7-e637941585d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:41:28.089336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.089303 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b54299-a944-4225-81d7-e637941585d8-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:28.089336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.089330 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jv9t\" (UniqueName: \"kubernetes.io/projected/d0b54299-a944-4225-81d7-e637941585d8-kube-api-access-2jv9t\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:28.089336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.089340 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b54299-a944-4225-81d7-e637941585d8-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:41:28.823914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.823878 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" Jul 09 22:41:28.824349 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.823875 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg" event={"ID":"d0b54299-a944-4225-81d7-e637941585d8","Type":"ContainerDied","Data":"638ba5c21f9809560ee678936a50dff7414f3f8c9d0b9ab6e1c66de08da3fdd0"} Jul 09 22:41:28.824349 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.823992 2575 scope.go:117] "RemoveContainer" containerID="8e118310f008482424fbaa3b3d3609176d26d5bce4a09b19398f2c1a45e8d9ec" Jul 09 22:41:28.848346 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.848317 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:28.853107 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:28.853084 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fdfdaecd-5bd784f4bb-k49bg"] Jul 09 22:41:29.192384 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:29.192344 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b54299-a944-4225-81d7-e637941585d8" path="/var/lib/kubelet/pods/d0b54299-a944-4225-81d7-e637941585d8/volumes" Jul 09 22:41:32.323710 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:32.323683 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:38.042872 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:38.042835 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:38.109303 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:38.109266 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:43.329670 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:43.329633 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:43.601877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:43.601795 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:44.780317 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.780278 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:44.856481 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.856451 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:41:44.857143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.857120 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b54299-a944-4225-81d7-e637941585d8" containerName="maas-api" Jul 09 22:41:44.857143 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.857143 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b54299-a944-4225-81d7-e637941585d8" containerName="maas-api" Jul 09 22:41:44.857336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.857260 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b54299-a944-4225-81d7-e637941585d8" containerName="maas-api" Jul 09 22:41:44.859238 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.859210 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:44.860542 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.860523 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:44.863518 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.863493 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e08843d5-serving-cert\"" Jul 09 22:41:44.870619 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.870597 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:41:44.929641 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.929608 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:44.951466 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.951429 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf2pg\" (UniqueName: \"kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:44.951626 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.951586 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:44.951735 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:44.951707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.053252 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.053167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.053252 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.053226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf2pg\" (UniqueName: \"kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.053447 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:41:45.053309 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-serving-cert: secret "maas-api-e2e-derive-e08843d5-serving-cert" not found Jul 09 22:41:45.053447 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:41:45.053386 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls podName:7d07e9b8-0596-4d71-ab89-94d35108f90c nodeName:}" failed. No retries permitted until 2026-07-09 22:41:45.553366766 +0000 UTC m=+1874.244940797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls") pod "maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" (UID: "7d07e9b8-0596-4d71-ab89-94d35108f90c") : secret "maas-api-e2e-derive-e08843d5-serving-cert" not found Jul 09 22:41:45.053447 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.053318 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.054026 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.054006 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.062515 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.062491 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf2pg\" (UniqueName: \"kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.204303 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.204266 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:45.368317 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.368221 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:45.465292 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.465258 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:45.559105 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.559073 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.561254 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.561235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") pod \"maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.774575 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.774540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:45.913130 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:45.913090 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:41:45.916199 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:41:45.916173 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d07e9b8_0596_4d71_ab89_94d35108f90c.slice/crio-951e454bc346c9cd01ada49c0145c9c34d86f2562268d505dbd1e7c4066fea6e WatchSource:0}: Error finding container 951e454bc346c9cd01ada49c0145c9c34d86f2562268d505dbd1e7c4066fea6e: Status 404 returned error can't find the container with id 951e454bc346c9cd01ada49c0145c9c34d86f2562268d505dbd1e7c4066fea6e Jul 09 22:41:46.918739 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:46.918700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" event={"ID":"7d07e9b8-0596-4d71-ab89-94d35108f90c","Type":"ContainerStarted","Data":"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8"} Jul 09 22:41:46.919149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:46.918762 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" event={"ID":"7d07e9b8-0596-4d71-ab89-94d35108f90c","Type":"ContainerStarted","Data":"951e454bc346c9cd01ada49c0145c9c34d86f2562268d505dbd1e7c4066fea6e"} Jul 09 22:41:46.919149 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:46.918791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:41:46.950024 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:46.949978 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" podStartSLOduration=2.4924897599999998 podStartE2EDuration="2.949963713s" totalImagesPullingTime="457.473953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:41:46.948041059 +0000 UTC m=+1875.639615116" watchObservedRunningTime="2026-07-09 22:41:46.949963713 +0000 UTC m=+1875.641537768" Jul 09 22:41:49.573110 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:49.573054 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:49.669922 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:49.669879 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:52.806781 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:52.806750 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:41:52.929518 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:41:52.929495 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:42:46.011022 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:46.010987 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:46.111400 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:46.111367 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:49.278061 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:49.278023 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:49.378265 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:49.378227 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:55.266189 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.266155 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:42:55.266714 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.266411 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" podUID="7d07e9b8-0596-4d71-ab89-94d35108f90c" containerName="maas-api" containerID="cri-o://0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8" gracePeriod=30 Jul 09 22:42:55.292042 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.292014 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:55.525968 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.525880 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:55.543626 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.543604 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:42:55.619960 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.619926 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca\") pod \"7d07e9b8-0596-4d71-ab89-94d35108f90c\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " Jul 09 22:42:55.620127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.620064 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf2pg\" (UniqueName: \"kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg\") pod \"7d07e9b8-0596-4d71-ab89-94d35108f90c\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " Jul 09 22:42:55.620127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.620104 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") pod \"7d07e9b8-0596-4d71-ab89-94d35108f90c\" (UID: \"7d07e9b8-0596-4d71-ab89-94d35108f90c\") " Jul 09 22:42:55.620343 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.620314 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d07e9b8-0596-4d71-ab89-94d35108f90c" (UID: "7d07e9b8-0596-4d71-ab89-94d35108f90c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:42:55.623450 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.623422 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d07e9b8-0596-4d71-ab89-94d35108f90c" (UID: "7d07e9b8-0596-4d71-ab89-94d35108f90c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:42:55.623560 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.623429 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg" (OuterVolumeSpecName: "kube-api-access-xf2pg") pod "7d07e9b8-0596-4d71-ab89-94d35108f90c" (UID: "7d07e9b8-0596-4d71-ab89-94d35108f90c"). InnerVolumeSpecName "kube-api-access-xf2pg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:42:55.721480 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.721451 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf2pg\" (UniqueName: \"kubernetes.io/projected/7d07e9b8-0596-4d71-ab89-94d35108f90c-kube-api-access-xf2pg\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:42:55.721480 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.721475 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d07e9b8-0596-4d71-ab89-94d35108f90c-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:42:55.721480 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.721485 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d07e9b8-0596-4d71-ab89-94d35108f90c-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:42:55.999943 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:55.999886 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:42:56.265800 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.265718 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d07e9b8-0596-4d71-ab89-94d35108f90c" containerID="0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8" exitCode=0 Jul 09 22:42:56.265800 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.265760 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" Jul 09 22:42:56.265800 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.265783 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" event={"ID":"7d07e9b8-0596-4d71-ab89-94d35108f90c","Type":"ContainerDied","Data":"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8"} Jul 09 22:42:56.266104 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.265834 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb" event={"ID":"7d07e9b8-0596-4d71-ab89-94d35108f90c","Type":"ContainerDied","Data":"951e454bc346c9cd01ada49c0145c9c34d86f2562268d505dbd1e7c4066fea6e"} Jul 09 22:42:56.266104 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.265860 2575 scope.go:117] "RemoveContainer" containerID="0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8" Jul 09 22:42:56.275160 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.275141 2575 scope.go:117] "RemoveContainer" containerID="0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8" Jul 09 22:42:56.275452 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:42:56.275431 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8\": container with ID starting with 0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8 not found: ID does not exist" containerID="0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8" Jul 09 22:42:56.275503 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.275462 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8"} err="failed to get container status \"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8\": rpc error: code = NotFound desc = could not find container \"0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8\": container with ID starting with 0bcdc4d4246a0d190e497a8e657feb99237bde9ef743415efcc4b7e74e814ed8 not found: ID does not exist" Jul 09 22:42:56.290147 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.290120 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:42:56.300269 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:56.300251 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e08843d5-5787d5f5b5-v55cb"] Jul 09 22:42:57.192209 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:42:57.192175 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d07e9b8-0596-4d71-ab89-94d35108f90c" path="/var/lib/kubelet/pods/7d07e9b8-0596-4d71-ab89-94d35108f90c/volumes" Jul 09 22:43:00.069236 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:00.069180 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:00.172475 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:00.172442 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:01.723311 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:01.723273 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:01.823005 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:01.822972 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:02.260637 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:02.260600 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:02.350794 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:02.350761 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:03.913878 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:03.913844 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:10.305168 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:10.305123 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:10.426249 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:10.426214 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:10.514796 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:10.514758 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:10.627801 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:10.627713 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:11.059909 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:11.059847 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:37.018565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:37.018527 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:42.994771 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:42.994738 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:43.233233 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:43.233195 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:43.345361 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:43.345282 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:43:59.099127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:43:59.099091 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:05.256936 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:05.256883 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:05.314914 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:05.314862 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:05.405284 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:05.405235 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:05.510348 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:05.510264 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:15.977348 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:15.977309 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:16.821027 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:16.820995 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:22.876711 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:22.876673 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:22.987813 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:22.987768 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:23.101420 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:23.101381 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:23.199562 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:23.199531 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:23.962982 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:23.962949 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:24.056984 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:24.056950 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:24.164220 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:24.164184 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:25.728788 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:25.728751 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:31.808830 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:31.808796 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:31.907570 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:31.907538 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:32.514111 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:32.514079 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:38.399322 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:38.399282 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:38.462705 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:38.462672 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:38.645229 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:38.645194 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:38.967561 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:38.967526 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:39.185678 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:39.185635 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.027864 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.027819 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.100398 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.100367 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:44:42.100948 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.100933 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d07e9b8-0596-4d71-ab89-94d35108f90c" containerName="maas-api" Jul 09 22:44:42.101012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.100950 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d07e9b8-0596-4d71-ab89-94d35108f90c" containerName="maas-api" Jul 09 22:44:42.101110 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.101098 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d07e9b8-0596-4d71-ab89-94d35108f90c" containerName="maas-api" Jul 09 22:44:42.104563 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.104544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.107099 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.107079 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-67cb547f-serving-cert\"" Jul 09 22:44:42.112309 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.111958 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:44:42.115242 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.115217 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.182416 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.182385 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68qbh\" (UniqueName: \"kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.182561 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.182446 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.182561 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.182499 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.188740 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.188712 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.283526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.283449 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.283526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.283504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.283736 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.283626 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68qbh\" (UniqueName: \"kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.284323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.284303 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.285925 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.285887 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.297689 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.297667 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68qbh\" (UniqueName: \"kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh\") pod \"maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.301945 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.301918 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.416305 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.416278 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:42.552729 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.552701 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:44:42.555958 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:44:42.555926 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c158a0_a85a_4965_9ff1_436d0d1bd518.slice/crio-5d1ac2691559cccaac706914d597754817f6e0a79729e6b996b2e09ce97df48a WatchSource:0}: Error finding container 5d1ac2691559cccaac706914d597754817f6e0a79729e6b996b2e09ce97df48a: Status 404 returned error can't find the container with id 5d1ac2691559cccaac706914d597754817f6e0a79729e6b996b2e09ce97df48a Jul 09 22:44:42.704936 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.704878 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:42.800003 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.799967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" event={"ID":"70c158a0-a85a-4965-9ff1-436d0d1bd518","Type":"ContainerStarted","Data":"5d1ac2691559cccaac706914d597754817f6e0a79729e6b996b2e09ce97df48a"} Jul 09 22:44:42.804184 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:42.804119 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:43.806281 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:43.806245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" event={"ID":"70c158a0-a85a-4965-9ff1-436d0d1bd518","Type":"ContainerStarted","Data":"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1"} Jul 09 22:44:43.806744 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:43.806322 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:46.477916 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:46.477862 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:47.183273 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:47.183236 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:47.368295 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:47.368263 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:47.568315 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:47.568234 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:49.817507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:49.817479 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:44:49.836357 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:49.836312 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" podStartSLOduration=7.368033789 podStartE2EDuration="7.83629745s" totalImagesPullingTime="468.263661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:44:43.824888317 +0000 UTC m=+2052.516462371" watchObservedRunningTime="2026-07-09 22:44:49.83629745 +0000 UTC m=+2058.527871502" Jul 09 22:44:53.534121 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:53.534021 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:53.797871 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:53.797779 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:44:53.894625 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:44:53.894596 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:04.655393 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.655363 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:45:04.655859 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.655604 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerName="maas-api" containerID="cri-o://b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1" gracePeriod=30 Jul 09 22:45:04.687767 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.687741 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:04.812678 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.812644 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.52:8443/health\": dial tcp 10.134.0.52:8443: connect: connection refused" Jul 09 22:45:04.904949 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.904924 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:45:04.918923 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.918856 2575 generic.go:356] "Generic (PLEG): container finished" podID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerID="b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1" exitCode=0 Jul 09 22:45:04.918923 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.918914 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" Jul 09 22:45:04.919038 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.918887 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" event={"ID":"70c158a0-a85a-4965-9ff1-436d0d1bd518","Type":"ContainerDied","Data":"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1"} Jul 09 22:45:04.919038 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.919032 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b" event={"ID":"70c158a0-a85a-4965-9ff1-436d0d1bd518","Type":"ContainerDied","Data":"5d1ac2691559cccaac706914d597754817f6e0a79729e6b996b2e09ce97df48a"} Jul 09 22:45:04.919104 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.919059 2575 scope.go:117] "RemoveContainer" containerID="b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1" Jul 09 22:45:04.928544 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.928527 2575 scope.go:117] "RemoveContainer" containerID="b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1" Jul 09 22:45:04.928779 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:45:04.928761 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1\": container with ID starting with b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1 not found: ID does not exist" containerID="b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1" Jul 09 22:45:04.928850 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.928787 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1"} err="failed to get container status \"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1\": rpc error: code = NotFound desc = could not find container \"b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1\": container with ID starting with b925a41756c65ece75af15c2a8f30ab728f7f15be5b8dbba583fb051a8b5cce1 not found: ID does not exist" Jul 09 22:45:04.996401 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.996377 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68qbh\" (UniqueName: \"kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh\") pod \"70c158a0-a85a-4965-9ff1-436d0d1bd518\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " Jul 09 22:45:04.996514 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.996464 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca\") pod \"70c158a0-a85a-4965-9ff1-436d0d1bd518\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " Jul 09 22:45:04.996555 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.996526 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls\") pod \"70c158a0-a85a-4965-9ff1-436d0d1bd518\" (UID: \"70c158a0-a85a-4965-9ff1-436d0d1bd518\") " Jul 09 22:45:04.996821 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.996798 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70c158a0-a85a-4965-9ff1-436d0d1bd518" (UID: "70c158a0-a85a-4965-9ff1-436d0d1bd518"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:45:04.998410 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.998392 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70c158a0-a85a-4965-9ff1-436d0d1bd518" (UID: "70c158a0-a85a-4965-9ff1-436d0d1bd518"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:45:04.998502 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:04.998487 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh" (OuterVolumeSpecName: "kube-api-access-68qbh") pod "70c158a0-a85a-4965-9ff1-436d0d1bd518" (UID: "70c158a0-a85a-4965-9ff1-436d0d1bd518"). InnerVolumeSpecName "kube-api-access-68qbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:45:05.097106 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:05.097083 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68qbh\" (UniqueName: \"kubernetes.io/projected/70c158a0-a85a-4965-9ff1-436d0d1bd518-kube-api-access-68qbh\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:45:05.097106 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:05.097105 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c158a0-a85a-4965-9ff1-436d0d1bd518-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:45:05.097260 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:05.097114 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c158a0-a85a-4965-9ff1-436d0d1bd518-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:45:05.245324 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:05.245297 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:45:05.255710 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:05.255683 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67cb547f-6f8457d7cb-h5h8b"] Jul 09 22:45:07.192320 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:07.192286 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" path="/var/lib/kubelet/pods/70c158a0-a85a-4965-9ff1-436d0d1bd518/volumes" Jul 09 22:45:08.753786 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:08.753750 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:09.246954 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:09.246918 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:15.134116 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:15.134083 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:15.188720 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:15.188689 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:19.901841 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:19.901795 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:20.132216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:20.132183 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:20.260478 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:20.260373 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:20.357363 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:20.357334 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:20.466705 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:20.466672 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:26.260184 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:26.260138 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:26.380518 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:26.380484 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:26.466942 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:26.466882 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:26.626797 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:26.626703 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:32.404853 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.404818 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:32.504578 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.504539 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:32.517798 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.517768 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:45:32.519602 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.519582 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:45:32.524444 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.524426 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:45:32.524950 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.524930 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:45:32.526520 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.526501 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:45:32.531014 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:32.530998 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:45:39.135362 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:39.135323 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:39.449010 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:39.448974 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:39.580822 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:39.580789 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:42.483416 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.483385 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:42.571312 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.571279 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:42.722408 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.722370 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:45:42.722981 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.722963 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerName="maas-api" Jul 09 22:45:42.723045 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.722984 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerName="maas-api" Jul 09 22:45:42.723152 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.723137 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c158a0-a85a-4965-9ff1-436d0d1bd518" containerName="maas-api" Jul 09 22:45:42.726612 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.726596 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.729402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.729383 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-eb6719bd-serving-cert\"" Jul 09 22:45:42.738248 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.738196 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:45:42.755316 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.755286 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:42.849045 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.849010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.849217 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.849067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd74k\" (UniqueName: \"kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.849217 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.849113 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.949629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.949598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd74k\" (UniqueName: \"kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.949629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.949630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.949831 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.949699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.949831 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:45:42.949818 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-serving-cert: secret "maas-api-e2e-mt-eb6719bd-serving-cert" not found Jul 09 22:45:42.949975 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:45:42.949924 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls podName:a66088b0-5467-43e6-97b6-d0de2422bbfb nodeName:}" failed. No retries permitted until 2026-07-09 22:45:43.449876946 +0000 UTC m=+2112.141450986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls") pod "maas-api-e2e-mt-eb6719bd-646646948b-72v8m" (UID: "a66088b0-5467-43e6-97b6-d0de2422bbfb") : secret "maas-api-e2e-mt-eb6719bd-serving-cert" not found Jul 09 22:45:42.950391 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.950373 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:42.962225 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:42.962205 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd74k\" (UniqueName: \"kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:43.235626 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.235587 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:43.362519 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.362489 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:43.455100 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.455067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:43.457514 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.457489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") pod \"maas-api-e2e-mt-eb6719bd-646646948b-72v8m\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:43.467884 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.467830 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:43.638980 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.638857 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:43.781475 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.781443 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:45:43.783775 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:45:43.783750 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66088b0_5467_43e6_97b6_d0de2422bbfb.slice/crio-904d3aeb35f4ebb4cf2edc686e3dadde7935cc970ca4925d1b6a8913a970dd79 WatchSource:0}: Error finding container 904d3aeb35f4ebb4cf2edc686e3dadde7935cc970ca4925d1b6a8913a970dd79: Status 404 returned error can't find the container with id 904d3aeb35f4ebb4cf2edc686e3dadde7935cc970ca4925d1b6a8913a970dd79 Jul 09 22:45:43.785804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:43.785785 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:45:44.117823 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:44.117779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" event={"ID":"a66088b0-5467-43e6-97b6-d0de2422bbfb","Type":"ContainerStarted","Data":"904d3aeb35f4ebb4cf2edc686e3dadde7935cc970ca4925d1b6a8913a970dd79"} Jul 09 22:45:45.123797 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:45.123760 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" event={"ID":"a66088b0-5467-43e6-97b6-d0de2422bbfb","Type":"ContainerStarted","Data":"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2"} Jul 09 22:45:45.124234 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:45.123918 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:45:45.142324 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:45.142284 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" podStartSLOduration=2.551302614 podStartE2EDuration="3.142270018s" totalImagesPullingTime="590.967404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:45:45.141066586 +0000 UTC m=+2113.832640652" watchObservedRunningTime="2026-07-09 22:45:45.142270018 +0000 UTC m=+2113.833844070" Jul 09 22:45:46.801556 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:46.801513 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:45:51.137187 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:45:51.137159 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:46:12.713731 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:46:12.713684 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:46:14.156272 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:46:14.156232 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:46:19.404579 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:46:19.404539 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:01.106944 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.106911 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:47:01.107465 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.107140 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerName="maas-api" containerID="cri-o://9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2" gracePeriod=30 Jul 09 22:47:01.132400 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.132367 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.53:8443/health\": dial tcp 10.134.0.53:8443: connect: connection refused" Jul 09 22:47:01.136942 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.136911 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:01.361085 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.361023 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:47:01.478543 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.478513 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca\") pod \"a66088b0-5467-43e6-97b6-d0de2422bbfb\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " Jul 09 22:47:01.478721 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.478620 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") pod \"a66088b0-5467-43e6-97b6-d0de2422bbfb\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " Jul 09 22:47:01.478791 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.478718 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd74k\" (UniqueName: \"kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k\") pod \"a66088b0-5467-43e6-97b6-d0de2422bbfb\" (UID: \"a66088b0-5467-43e6-97b6-d0de2422bbfb\") " Jul 09 22:47:01.478849 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.478820 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a66088b0-5467-43e6-97b6-d0de2422bbfb" (UID: "a66088b0-5467-43e6-97b6-d0de2422bbfb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:47:01.479041 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.479024 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a66088b0-5467-43e6-97b6-d0de2422bbfb-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:47:01.480445 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.480421 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a66088b0-5467-43e6-97b6-d0de2422bbfb" (UID: "a66088b0-5467-43e6-97b6-d0de2422bbfb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:47:01.480546 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.480509 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k" (OuterVolumeSpecName: "kube-api-access-vd74k") pod "a66088b0-5467-43e6-97b6-d0de2422bbfb" (UID: "a66088b0-5467-43e6-97b6-d0de2422bbfb"). InnerVolumeSpecName "kube-api-access-vd74k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:47:01.523856 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.523825 2575 generic.go:356] "Generic (PLEG): container finished" podID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerID="9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2" exitCode=0 Jul 09 22:47:01.523972 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.523862 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" Jul 09 22:47:01.523972 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.523934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" event={"ID":"a66088b0-5467-43e6-97b6-d0de2422bbfb","Type":"ContainerDied","Data":"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2"} Jul 09 22:47:01.524052 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.523979 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m" event={"ID":"a66088b0-5467-43e6-97b6-d0de2422bbfb","Type":"ContainerDied","Data":"904d3aeb35f4ebb4cf2edc686e3dadde7935cc970ca4925d1b6a8913a970dd79"} Jul 09 22:47:01.524052 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.524000 2575 scope.go:117] "RemoveContainer" containerID="9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2" Jul 09 22:47:01.533062 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.533044 2575 scope.go:117] "RemoveContainer" containerID="9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2" Jul 09 22:47:01.533277 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:47:01.533261 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2\": container with ID starting with 9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2 not found: ID does not exist" containerID="9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2" Jul 09 22:47:01.533317 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.533286 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2"} err="failed to get container status \"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2\": rpc error: code = NotFound desc = could not find container \"9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2\": container with ID starting with 9e364dbbe28c1b9afee581506319a177e26ed1eeb74a774e9c484185035bacc2 not found: ID does not exist" Jul 09 22:47:01.546484 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.546458 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:47:01.549907 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.549878 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eb6719bd-646646948b-72v8m"] Jul 09 22:47:01.579803 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.579781 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a66088b0-5467-43e6-97b6-d0de2422bbfb-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:47:01.579803 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:01.579801 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd74k\" (UniqueName: \"kubernetes.io/projected/a66088b0-5467-43e6-97b6-d0de2422bbfb-kube-api-access-vd74k\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:47:03.193567 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:03.193524 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" path="/var/lib/kubelet/pods/a66088b0-5467-43e6-97b6-d0de2422bbfb/volumes" Jul 09 22:47:05.742430 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:05.742389 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:10.832935 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:10.832881 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:46.956964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:46.956926 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:48.167213 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:48.167175 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:53.760963 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:53.760872 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:47:53.826289 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:47:53.826257 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:48:01.229339 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:48:01.229284 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:48:01.328582 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:48:01.328555 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:50:07.369553 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:07.369515 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:50:32.548156 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.548127 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:50:32.550005 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.549985 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:50:32.554439 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.554418 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:50:32.555323 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.555303 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:50:32.557049 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.557031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:50:32.561700 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:50:32.561683 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:51:02.027543 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:02.027445 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:02.244792 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:02.244754 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:03.392175 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:03.392142 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:03.494988 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:03.494958 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:03.588868 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:03.588834 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:04.345820 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:04.345790 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:04.437650 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:04.437617 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:05.853291 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:05.853254 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:11.643676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:11.643625 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:11.709676 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:11.709641 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:12.334243 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:12.334153 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:18.481023 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:18.480989 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:19.731672 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:19.731617 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:29.288204 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:29.288171 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:29.564443 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:29.564358 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:48.518353 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:48.518321 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:50.520537 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:50.520503 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:56.287507 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:56.287473 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:56.387334 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:56.387299 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:56.647923 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:56.647883 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:57.000141 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:57.000051 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:51:57.441511 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:51:57.441474 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:00.505989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.505943 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:00.605183 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.605147 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:00.726315 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.726281 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:52:00.727469 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.727443 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerName="maas-api" Jul 09 22:52:00.727619 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.727608 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerName="maas-api" Jul 09 22:52:00.728852 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.728830 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66088b0-5467-43e6-97b6-d0de2422bbfb" containerName="maas-api" Jul 09 22:52:00.735690 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.735663 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.738105 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.738078 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:52:00.738505 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.738488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-288815-serving-cert\"" Jul 09 22:52:00.765385 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.765319 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:00.862750 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.862718 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brlv\" (UniqueName: \"kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.862964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.862826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.862964 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.862853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.964124 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.964085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.964124 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.964130 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.964330 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.964195 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brlv\" (UniqueName: \"kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.964700 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.964673 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.965835 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.965806 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:00.966739 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.966719 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:00.972476 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:00.972457 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brlv\" (UniqueName: \"kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv\") pod \"maas-api-e2e-inf-a-288815-846dd864fd-g9ztm\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:01.050086 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:01.050017 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:01.186012 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:01.185977 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:52:01.190399 ip-10-0-130-127 kubenswrapper[2575]: W0709 22:52:01.190374 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda548f8e1_f00c_42f8_a253_9b1c83fc130d.slice/crio-386642dcd46806f5376d6b4f7d35195b1b7bf9c5cd2c6d9d3bc8778ca48cab2f WatchSource:0}: Error finding container 386642dcd46806f5376d6b4f7d35195b1b7bf9c5cd2c6d9d3bc8778ca48cab2f: Status 404 returned error can't find the container with id 386642dcd46806f5376d6b4f7d35195b1b7bf9c5cd2c6d9d3bc8778ca48cab2f Jul 09 22:52:01.192107 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:01.192089 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:52:01.243922 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:01.243873 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:01.344830 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:01.344746 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:02.007071 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:02.007041 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" event={"ID":"a548f8e1-f00c-42f8-a253-9b1c83fc130d","Type":"ContainerStarted","Data":"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef"} Jul 09 22:52:02.007417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:02.007090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" event={"ID":"a548f8e1-f00c-42f8-a253-9b1c83fc130d","Type":"ContainerStarted","Data":"386642dcd46806f5376d6b4f7d35195b1b7bf9c5cd2c6d9d3bc8778ca48cab2f"} Jul 09 22:52:02.007417 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:02.007149 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:02.025140 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:02.025091 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" podStartSLOduration=1.596332358 podStartE2EDuration="2.025075791s" totalImagesPullingTime="428.743433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:52:02.024127493 +0000 UTC m=+2490.715701547" watchObservedRunningTime="2026-07-09 22:52:02.025075791 +0000 UTC m=+2490.716649843" Jul 09 22:52:04.912434 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:04.912398 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:05.250809 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:05.250720 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:05.599054 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:05.598961 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:05.699018 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:05.698987 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:05.799974 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:05.799935 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:08.018565 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:08.018538 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:52:08.398177 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:08.398138 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:08.532706 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:08.532670 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:08.631989 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:08.631956 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:08.940301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:08.940265 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:09.099438 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:09.099398 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:09.198977 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:09.198881 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:12.882275 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:12.882244 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:22.001671 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:22.001633 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:22.100622 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:22.100581 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:30.043031 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:30.042945 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:30.145057 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:30.145018 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:52:32.329187 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:52:32.329152 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:02.071232 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:02.071193 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:02.171867 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:02.171830 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:11.324478 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:11.324439 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:31.872815 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:31.872777 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:32.857041 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:32.857002 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:33.449998 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:33.449963 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:33.558126 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:33.558090 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:43.465577 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.465537 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:53:43.466068 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.465822 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" podUID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" containerName="maas-api" containerID="cri-o://acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef" gracePeriod=30 Jul 09 22:53:43.496577 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.496551 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:43.713072 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.713045 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:53:43.842040 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.841963 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls\") pod \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " Jul 09 22:53:43.842192 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.842051 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca\") pod \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " Jul 09 22:53:43.842192 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.842127 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4brlv\" (UniqueName: \"kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv\") pod \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\" (UID: \"a548f8e1-f00c-42f8-a253-9b1c83fc130d\") " Jul 09 22:53:43.842459 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.842429 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a548f8e1-f00c-42f8-a253-9b1c83fc130d" (UID: "a548f8e1-f00c-42f8-a253-9b1c83fc130d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:53:43.844132 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.844110 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a548f8e1-f00c-42f8-a253-9b1c83fc130d" (UID: "a548f8e1-f00c-42f8-a253-9b1c83fc130d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:53:43.844214 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.844174 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv" (OuterVolumeSpecName: "kube-api-access-4brlv") pod "a548f8e1-f00c-42f8-a253-9b1c83fc130d" (UID: "a548f8e1-f00c-42f8-a253-9b1c83fc130d"). InnerVolumeSpecName "kube-api-access-4brlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:53:43.943367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.943338 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a548f8e1-f00c-42f8-a253-9b1c83fc130d-openshift-service-ca\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:53:43.943367 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.943366 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4brlv\" (UniqueName: \"kubernetes.io/projected/a548f8e1-f00c-42f8-a253-9b1c83fc130d-kube-api-access-4brlv\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:53:43.943526 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:43.943376 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a548f8e1-f00c-42f8-a253-9b1c83fc130d-maas-api-tls\") on node \"ip-10-0-130-127.ec2.internal\" DevicePath \"\"" Jul 09 22:53:44.511685 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.511637 2575 generic.go:356] "Generic (PLEG): container finished" podID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" containerID="acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef" exitCode=0 Jul 09 22:53:44.511685 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.511687 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" Jul 09 22:53:44.512163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.511721 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" event={"ID":"a548f8e1-f00c-42f8-a253-9b1c83fc130d","Type":"ContainerDied","Data":"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef"} Jul 09 22:53:44.512163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.511774 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm" event={"ID":"a548f8e1-f00c-42f8-a253-9b1c83fc130d","Type":"ContainerDied","Data":"386642dcd46806f5376d6b4f7d35195b1b7bf9c5cd2c6d9d3bc8778ca48cab2f"} Jul 09 22:53:44.512163 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.511799 2575 scope.go:117] "RemoveContainer" containerID="acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef" Jul 09 22:53:44.521545 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.521521 2575 scope.go:117] "RemoveContainer" containerID="acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef" Jul 09 22:53:44.521776 ip-10-0-130-127 kubenswrapper[2575]: E0709 22:53:44.521758 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef\": container with ID starting with acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef not found: ID does not exist" containerID="acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef" Jul 09 22:53:44.521825 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.521785 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef"} err="failed to get container status \"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef\": rpc error: code = NotFound desc = could not find container \"acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef\": container with ID starting with acb835b09fff10f3a01423acf402a721e62a0395d8e284e36d3bd4baedf0c5ef not found: ID does not exist" Jul 09 22:53:44.535662 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.535631 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:53:44.539722 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:44.539702 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-288815-846dd864fd-g9ztm"] Jul 09 22:53:45.192289 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:45.192260 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" path="/var/lib/kubelet/pods/a548f8e1-f00c-42f8-a253-9b1c83fc130d/volumes" Jul 09 22:53:48.052809 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:48.052776 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:53.907947 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:53.907845 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:54.117402 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:54.117361 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:58.205800 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:58.205771 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:58.710301 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:58.710263 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:58.942170 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:58.942141 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:59.604372 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:59.604334 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:53:59.705392 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:53:59.705359 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:09.536741 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:09.536705 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:14.160549 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:14.160508 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:20.072362 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:20.072322 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:20.141156 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:20.141105 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:24.848145 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:24.848111 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:25.086939 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:25.086886 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:25.434838 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:25.434803 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:25.905885 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:25.905845 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:26.117347 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:26.117314 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:29.193662 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:29.193614 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:29.279974 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:29.279943 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:29.448288 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:29.448197 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:29.642913 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:29.642857 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:30.086695 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:30.086658 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:30.194522 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:30.194482 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:33.678502 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:33.678468 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:34.140246 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:34.140157 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:34.503216 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:34.503187 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:34.662646 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:34.662620 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.267009 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.266970 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.354359 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.354326 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.451133 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.451098 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.550629 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.550538 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.873805 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.873726 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:37.976261 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:37.976228 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:39.989852 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:39.989820 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:41.785181 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:41.785147 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:50.320599 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:50.320562 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:50.420722 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:50.420689 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:58.796708 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:58.796666 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:54:58.900413 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:54:58.900385 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:00.446675 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:00.446639 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:00.554877 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:00.554838 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:00.714193 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:00.714108 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:04.892984 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:04.892940 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:10.893538 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:10.893502 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:10.974534 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:10.974497 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:15.545804 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:15.545770 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:15.781235 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:15.781200 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:16.421243 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:16.421212 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:16.522990 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:16.522955 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:26.356630 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:26.356545 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:31.075720 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:31.075682 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:32.579330 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.579304 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:55:32.581081 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.581059 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:55:32.585386 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.585367 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:55:32.587280 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.587264 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 22:55:32.589065 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.589047 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 22:55:32.593440 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:32.593424 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 22:55:36.903141 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:36.903102 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:36.967321 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:36.967278 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:41.727146 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:41.727107 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:41.947245 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:41.947204 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:42.337247 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:42.337190 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:42.645135 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:42.645104 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:42.995215 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:42.995126 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:45.608318 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:45.608277 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:45.707683 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:45.707644 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:45.849275 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:45.849244 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:45.946653 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:45.946618 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:46.425824 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:46.425796 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:46.524245 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:46.524212 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:50.125335 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:50.125301 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:50.512205 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:50.512176 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:50.741127 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:50.741094 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:51.152180 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:51.152143 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:53.655677 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:53.655649 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:53.743842 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:53.743803 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:53.843751 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:53.843720 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:53.983336 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:53.983245 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:54.316659 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:54.316569 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:54.417084 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:54.417050 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:55:58.159866 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:55:58.159826 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:08.079906 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:08.079832 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:08.182544 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:08.182492 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:13.657473 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:13.657439 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:15.149861 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:15.149827 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:15.251260 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:15.251227 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:56:38.025380 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:56:38.025340 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:57:52.837545 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:57:52.837510 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:58:02.806279 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:58:02.806235 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:58:02.906814 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:58:02.906778 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:58:42.301932 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:58:42.301831 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:58:43.170136 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:58:43.170101 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:58:47.102874 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:58:47.102836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:59:11.698500 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:59:11.698459 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:59:39.202838 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:59:39.202800 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:59:48.603517 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:59:48.603477 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 22:59:49.527882 ip-10-0-130-127 kubenswrapper[2575]: I0709 22:59:49.527845 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:00:07.243291 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:07.243259 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:00:15.082496 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:15.082461 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:00:15.179965 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:15.179922 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:00:30.712749 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:30.712717 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:00:32.609540 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.609504 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 23:00:32.611284 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.611262 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 23:00:32.615767 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.615749 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 23:00:32.618866 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.618847 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 23:00:32.620601 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.620580 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 23:00:32.625383 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:00:32.625367 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d6n4b_986ae43d-bda5-4c43-a126-3217ec5151a9/ovn-acl-logging/0.log" Jul 09 23:01:08.825228 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:08.825159 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:08.926964 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:08.926926 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:11.249252 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:11.249210 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:12.141723 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:12.141686 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:12.608454 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:12.608427 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:16.017335 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:16.017301 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:16.113458 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:16.113429 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:31.863645 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:31.863550 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:31.964623 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:31.964589 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:35.927955 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:35.927906 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:41.662361 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:41.662320 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:49.028627 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:49.028587 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:52.200771 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:52.200735 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:52.265395 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:52.265359 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:57.019054 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:57.019014 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:57.276093 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:57.275995 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:58.179346 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:58.179313 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:01:58.279655 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:01:58.279610 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:07.871910 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:07.871872 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:12.494658 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:12.494628 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:18.644111 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:18.644073 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:18.744150 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:18.744113 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:23.191323 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:23.191288 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:23.435445 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:23.435417 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:29.287833 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:29.287791 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b7cd97f87-gjkbm_cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4/manager/0.log" Jul 09 23:02:29.951754 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:29.951722 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf875db85-8s6z9_ebb7f9c8-3c6d-431b-a998-e12c29925513/copy-manifests/0.log" Jul 09 23:02:29.958006 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:29.957983 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf875db85-8s6z9_ebb7f9c8-3c6d-431b-a998-e12c29925513/manager/0.log" Jul 09 23:02:30.340744 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:30.340671 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-648645db76-nfjcf_1891e73f-7d7d-4e37-b026-a3c5278d0b1c/maas-api/0.log" Jul 09 23:02:32.138282 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:32.138251 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hsn56_e52bd149-2915-4219-92f4-ef20307a59b3/postgres/0.log" Jul 09 23:02:34.044753 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:34.044702 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vtrgc_e466afd8-dc85-4df7-9b0a-87c5be8955bc/registry-server/0.log" Jul 09 23:02:34.167127 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:34.167096 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jp6tt_c092e5c8-645a-40e7-9049-2246a4aa3d83/manager/0.log" Jul 09 23:02:34.419428 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:34.419383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-466pv_939f0861-a88d-4e00-ba28-0cfda0243fb3/manager/0.log" Jul 09 23:02:34.949694 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:34.949664 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-62zk9_941d1cf0-7d9c-4033-b423-93f02753bd43/discovery/0.log" Jul 09 23:02:35.072489 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:35.072444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff74c44b9-d9wr6_2592be26-7d07-4f5a-902d-214aefcd8cec/kube-auth-proxy/0.log" Jul 09 23:02:35.455406 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:35.455377 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k4mxq_b189cf51-52c4-45b5-bc13-cd7f506e50f6/payload-pre-processing/0.log" Jul 09 23:02:36.124327 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:36.124298 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-vnmv4_6cb620bb-31dc-41a5-9e4e-fb21d7e9b4d7/main/0.log" Jul 09 23:02:36.361157 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:36.361126 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-86dlh_4027d33b-a2f9-485d-8e43-b20d433bd18e/main/0.log" Jul 09 23:02:36.495456 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:36.495427 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmw46r_db7f7cdc-7167-4cff-8a08-dcd8d96b52b6/main/0.log" Jul 09 23:02:36.625753 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:36.625726 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8sr8c_33892e01-8822-4dac-8037-6cd49cc51ee5/main/0.log" Jul 09 23:02:41.593627 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:41.593591 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:41.649938 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:41.649880 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:02:43.516211 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:43.516179 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gddff_0a8756ff-0a32-454c-afdb-79d5ee1eb974/global-pull-secret-syncer/0.log" Jul 09 23:02:43.614828 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:43.614799 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" Jul 09 23:02:43.714282 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:43.714256 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-127.ec2.internal_7a92842bdfd3fd5ca29ad4095ae200ef/haproxy/0.log" Jul 09 23:02:49.854949 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:49.854914 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vtrgc_e466afd8-dc85-4df7-9b0a-87c5be8955bc/registry-server/0.log" Jul 09 23:02:50.047786 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:50.047748 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jp6tt_c092e5c8-645a-40e7-9049-2246a4aa3d83/manager/0.log" Jul 09 23:02:50.190056 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:50.190023 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-466pv_939f0861-a88d-4e00-ba28-0cfda0243fb3/manager/0.log" Jul 09 23:02:51.791713 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.791679 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/alertmanager/0.log" Jul 09 23:02:51.815411 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.815387 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/config-reloader/0.log" Jul 09 23:02:51.837325 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.837301 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/kube-rbac-proxy-web/0.log" Jul 09 23:02:51.858607 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.858587 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/kube-rbac-proxy/0.log" Jul 09 23:02:51.879704 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.879683 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/kube-rbac-proxy-metric/0.log" Jul 09 23:02:51.901672 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.901648 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/prom-label-proxy/0.log" Jul 09 23:02:51.924991 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.924952 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35691cf3-c17e-429d-9f0f-6347619e03e2/init-config-reloader/0.log" Jul 09 23:02:51.981421 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:51.981390 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/1.log" Jul 09 23:02:52.131160 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.131075 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-twgjb_84465f8d-067f-407e-9efb-aeec5b9d16c0/cluster-monitoring-operator/0.log" Jul 09 23:02:52.160673 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.160650 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-xj4rz_29fb3c25-e31d-43be-a93f-f6692ec798f0/kube-state-metrics/0.log" Jul 09 23:02:52.188309 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.188283 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-xj4rz_29fb3c25-e31d-43be-a93f-f6692ec798f0/kube-rbac-proxy-main/0.log" Jul 09 23:02:52.212260 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.212240 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-xj4rz_29fb3c25-e31d-43be-a93f-f6692ec798f0/kube-rbac-proxy-self/0.log" Jul 09 23:02:52.239430 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.239404 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d4d665b8-ds9hd_cfbb998a-a4d5-4a7b-b2d0-e93bdcaa5e50/metrics-server/0.log" Jul 09 23:02:52.299774 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.299748 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5sw4w_1e0eb537-921f-4d36-9c87-ca16c0f92207/node-exporter/0.log" Jul 09 23:02:52.328953 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.328930 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5sw4w_1e0eb537-921f-4d36-9c87-ca16c0f92207/kube-rbac-proxy/0.log" Jul 09 23:02:52.366179 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.366160 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5sw4w_1e0eb537-921f-4d36-9c87-ca16c0f92207/init-textfile/0.log" Jul 09 23:02:52.634095 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.634060 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9djpd_7e588d5f-07ca-4d38-aefc-7c225eceb236/kube-rbac-proxy-main/0.log" Jul 09 23:02:52.661278 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.661254 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9djpd_7e588d5f-07ca-4d38-aefc-7c225eceb236/kube-rbac-proxy-self/0.log" Jul 09 23:02:52.697031 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:52.697010 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9djpd_7e588d5f-07ca-4d38-aefc-7c225eceb236/openshift-state-metrics/0.log" Jul 09 23:02:53.514466 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.512883 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth"] Jul 09 23:02:53.514466 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.513925 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" containerName="maas-api" Jul 09 23:02:53.514466 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.513963 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" containerName="maas-api" Jul 09 23:02:53.514466 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.514243 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a548f8e1-f00c-42f8-a253-9b1c83fc130d" containerName="maas-api" Jul 09 23:02:53.519307 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.519284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.522378 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.522348 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5rfjs\"/\"default-dockercfg-dbb7w\"" Jul 09 23:02:53.522517 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.522348 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5rfjs\"/\"kube-root-ca.crt\"" Jul 09 23:02:53.522517 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.522381 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5rfjs\"/\"openshift-service-ca.crt\"" Jul 09 23:02:53.531707 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.531684 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth"] Jul 09 23:02:53.584207 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.584184 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-sys\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.584312 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.584211 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbwtj\" (UniqueName: \"kubernetes.io/projected/d2b358a2-c50f-4558-a73c-97426d14046b-kube-api-access-rbwtj\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.584312 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.584230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-lib-modules\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.584312 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.584301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-podres\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.584464 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.584332 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-proc\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685436 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685406 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-lib-modules\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685548 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-podres\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685548 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-proc\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685548 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-sys\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685696 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685552 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbwtj\" (UniqueName: \"kubernetes.io/projected/d2b358a2-c50f-4558-a73c-97426d14046b-kube-api-access-rbwtj\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685696 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685584 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-lib-modules\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685696 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685602 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-proc\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685696 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685616 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-sys\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.685696 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.685605 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2b358a2-c50f-4558-a73c-97426d14046b-podres\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.694355 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.694330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbwtj\" (UniqueName: \"kubernetes.io/projected/d2b358a2-c50f-4558-a73c-97426d14046b-kube-api-access-rbwtj\") pod \"perf-node-gather-daemonset-2hcth\" (UID: \"d2b358a2-c50f-4558-a73c-97426d14046b\") " pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.829727 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.829650 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:53.960409 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:53.960385 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth"] Jul 09 23:02:53.961782 ip-10-0-130-127 kubenswrapper[2575]: W0709 23:02:53.961758 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2b358a2_c50f_4558_a73c_97426d14046b.slice/crio-2bc4850e9f0a7c2929fff54458f659b751666348abba42eae3d1f0780662a6b4 WatchSource:0}: Error finding container 2bc4850e9f0a7c2929fff54458f659b751666348abba42eae3d1f0780662a6b4: Status 404 returned error can't find the container with id 2bc4850e9f0a7c2929fff54458f659b751666348abba42eae3d1f0780662a6b4 Jul 09 23:02:54.278455 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:54.278418 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" event={"ID":"d2b358a2-c50f-4558-a73c-97426d14046b","Type":"ContainerStarted","Data":"3ee09d37d9136f4eab4cbbecc743cdd1a3345903dcf29bf97bc0cb323795a8b7"} Jul 09 23:02:54.278623 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:54.278465 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" event={"ID":"d2b358a2-c50f-4558-a73c-97426d14046b","Type":"ContainerStarted","Data":"2bc4850e9f0a7c2929fff54458f659b751666348abba42eae3d1f0780662a6b4"} Jul 09 23:02:54.278623 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:54.278494 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:02:55.051035 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:55.051001 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/1.log" Jul 09 23:02:55.059829 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:55.059809 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lrvcq_e35b1e39-6bb4-417a-b8a6-a1560095a0c8/console-operator/2.log" Jul 09 23:02:55.521817 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:55.521788 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-88d96b4f6-7l5kr_e6197cf4-169c-4213-80ed-203cd6ba90ae/console/0.log" Jul 09 23:02:56.132280 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:56.132252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-kjjss_d2f0e289-c0c9-404f-b7a7-60c2a6741163/volume-data-source-validator/0.log" Jul 09 23:02:57.259838 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:57.259811 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b4w94_b8f0f8e8-a18a-450e-81af-c9385ea3c9a7/dns/0.log" Jul 09 23:02:57.280644 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:57.280604 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b4w94_b8f0f8e8-a18a-450e-81af-c9385ea3c9a7/kube-rbac-proxy/0.log" Jul 09 23:02:57.349771 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:57.349749 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7r2hf_075e707e-55d1-476f-b14c-4875d35f240d/dns-node-resolver/0.log" Jul 09 23:02:57.940051 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:57.939917 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-779764dffc-8l7hg_358db4b1-0adf-45db-a5ec-fc9e2f8aa446/registry/0.log" Jul 09 23:02:58.025784 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:58.025745 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sh9k4_5a3da10f-2ff6-4343-9843-cd58a6fd7339/node-ca/0.log" Jul 09 23:02:59.205517 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:59.205488 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-62zk9_941d1cf0-7d9c-4033-b423-93f02753bd43/discovery/0.log" Jul 09 23:02:59.227774 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:59.227756 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff74c44b9-d9wr6_2592be26-7d07-4f5a-902d-214aefcd8cec/kube-auth-proxy/0.log" Jul 09 23:02:59.352541 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:02:59.352510 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k4mxq_b189cf51-52c4-45b5-bc13-cd7f506e50f6/payload-pre-processing/0.log" Jul 09 23:03:00.173011 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:00.172979 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d479r_b45a8325-8a9b-41f0-ba02-0b31ddd5aea1/serve-healthcheck-canary/0.log" Jul 09 23:03:00.293094 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:00.293071 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" Jul 09 23:03:00.313391 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:00.313346 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5rfjs/perf-node-gather-daemonset-2hcth" podStartSLOduration=7.313331424 podStartE2EDuration="7.313331424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 23:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 23:02:54.295666077 +0000 UTC m=+3142.987240132" watchObservedRunningTime="2026-07-09 23:03:00.313331424 +0000 UTC m=+3149.004905513" Jul 09 23:03:00.779066 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:00.779037 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-8btqh_48090bc2-a6c4-4412-826a-cc4c65e24085/insights-operator/0.log" Jul 09 23:03:00.780130 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:00.780110 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-8btqh_48090bc2-a6c4-4412-826a-cc4c65e24085/insights-operator/1.log" Jul 09 23:03:01.009751 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:01.009725 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clgxj_bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec/kube-rbac-proxy/0.log" Jul 09 23:03:01.030117 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:01.030048 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clgxj_bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec/exporter/0.log" Jul 09 23:03:01.051390 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:01.051371 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clgxj_bdceb3e1-b99f-4a9e-b8cc-6db512d9f7ec/extractor/0.log" Jul 09 23:03:03.676469 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:03.676355 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:03:03.775926 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:03.775834 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8kktk_715df68e-6d7c-4334-8dc1-f8684bf5a1af/konnectivity-agent/0.log" limit=1024 Jul 09 23:03:03.854884 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:03.854858 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b7cd97f87-gjkbm_cacd91b7-8d35-429d-aab5-b7d6ec3ddfd4/manager/0.log" Jul 09 23:03:04.166998 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:04.166971 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf875db85-8s6z9_ebb7f9c8-3c6d-431b-a998-e12c29925513/manager/0.log" Jul 09 23:03:04.188803 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:04.188775 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cf875db85-8s6z9_ebb7f9c8-3c6d-431b-a998-e12c29925513/copy-manifests/0.log" Jul 09 23:03:04.664257 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:04.664230 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-62zk9_941d1cf0-7d9c-4033-b423-93f02753bd43/discovery/0.log" Jul 09 23:03:04.676281 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:04.676254 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff74c44b9-d9wr6_2592be26-7d07-4f5a-902d-214aefcd8cec/kube-auth-proxy/0.log" Jul 09 23:03:04.773579 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:04.773553 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k4mxq_b189cf51-52c4-45b5-bc13-cd7f506e50f6/payload-pre-processing/0.log" Jul 09 23:03:06.275678 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:06.275649 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-62zk9_941d1cf0-7d9c-4033-b423-93f02753bd43/discovery/0.log" Jul 09 23:03:06.352817 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:06.352787 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff74c44b9-d9wr6_2592be26-7d07-4f5a-902d-214aefcd8cec/kube-auth-proxy/0.log" Jul 09 23:03:06.572221 ip-10-0-130-127 kubenswrapper[2575]: I0709 23:03:06.572153 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k4mxq_b189cf51-52c4-45b5-bc13-cd7f506e50f6/payload-pre-processing/0.log"