Jul 15 23:47:24.827483 ip-10-0-129-211 systemd[1]: Starting Kubernetes Kubelet... Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.220478 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228488 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228601 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228608 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228611 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228615 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228618 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228621 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.328132 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228624 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.294836 ip-10-0-129-211 systemd[1]: Started Kubernetes Kubelet. Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228628 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228631 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228635 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228639 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228642 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228645 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228648 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228650 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228653 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228656 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228658 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228661 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228664 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228666 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228669 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228671 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228674 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228676 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.329330 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228679 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228682 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228685 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228689 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228692 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228694 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228697 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228703 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228705 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228708 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228713 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228715 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228718 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228721 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228723 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228726 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228729 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228731 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228735 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.330329 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228739 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228742 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228745 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228749 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228752 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228756 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228760 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228764 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228767 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228770 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228773 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228776 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228779 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228783 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228785 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228790 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228793 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228795 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228798 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.331003 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228801 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228804 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228807 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228810 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228812 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228816 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228818 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228821 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228824 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228826 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228829 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228831 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228834 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228838 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228841 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228844 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228847 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228849 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228852 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228855 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.331674 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228858 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228860 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228863 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228866 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228868 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228871 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228874 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228877 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228879 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228881 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228884 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228887 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228889 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228892 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.228894 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229618 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229627 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229630 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229633 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229636 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.332348 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229639 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229641 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229644 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229646 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229661 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229668 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229672 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229676 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229680 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229685 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229689 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229692 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229697 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229701 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229704 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229708 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229712 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229715 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229718 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.333270 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229720 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229723 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229727 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229731 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229734 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229738 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229742 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229746 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229749 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229754 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229757 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229761 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229765 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229769 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229773 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229777 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229781 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229785 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229791 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229795 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.333862 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229798 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229803 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229806 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229810 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229813 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229817 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229822 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229827 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229833 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229836 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229840 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229844 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229851 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229855 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229860 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229864 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229868 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229872 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229876 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229880 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.334485 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229884 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229888 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229892 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229896 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229900 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229903 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229907 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229911 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229916 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229920 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229926 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229929 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229931 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229935 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229938 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229943 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229947 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229951 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229955 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229962 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.335063 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229967 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229971 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229976 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229980 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229984 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229988 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229991 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229995 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.229999 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230003 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230006 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230010 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230014 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230018 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230022 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230630 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230639 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230644 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230648 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230653 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.335683 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230656 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230660 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230668 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230671 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230675 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230680 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230684 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230688 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230692 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230696 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230700 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230704 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230708 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230714 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230718 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230722 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230726 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230732 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230737 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.336232 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230741 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230745 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230749 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230753 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230757 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230760 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230762 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230766 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230770 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230774 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230777 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230781 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230785 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230789 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230792 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230798 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230801 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230807 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230811 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.336808 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230817 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230821 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230826 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230830 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230834 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230838 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230842 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230846 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230850 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230853 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230857 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230862 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230866 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230869 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230873 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230877 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230880 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230884 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230888 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230892 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.337325 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230895 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230899 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230903 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230907 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230911 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230914 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230919 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230923 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230928 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230934 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230937 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230941 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230945 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230948 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230952 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230956 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230960 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230964 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230966 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230970 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.337927 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230973 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230977 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230981 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230984 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230988 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230992 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.230996 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231000 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231004 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231007 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231011 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231014 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231018 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231022 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231025 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231029 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231128 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231137 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231147 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231152 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231160 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 23:47:25.338553 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231166 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231171 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231178 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231181 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231184 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231189 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231194 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231199 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231203 2580 flags.go:64] FLAG: --cgroup-root="" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231207 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231211 2580 flags.go:64] FLAG: --client-ca-file="" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231231 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231237 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231242 2580 flags.go:64] FLAG: --cluster-domain="" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231247 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231251 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231255 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231260 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231266 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231270 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231274 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231278 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231283 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231287 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 23:47:25.339190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231292 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231296 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231301 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231306 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231311 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231315 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231319 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231325 2580 flags.go:64] FLAG: --enable-server="true" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231331 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231339 2580 flags.go:64] FLAG: --event-burst="100" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231344 2580 flags.go:64] FLAG: --event-qps="50" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231348 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231353 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231357 2580 flags.go:64] FLAG: --eviction-hard="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231362 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231367 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231371 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231375 2580 flags.go:64] FLAG: --eviction-soft="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231380 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231384 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231388 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231392 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231396 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231400 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231404 2580 flags.go:64] FLAG: --feature-gates="" Jul 15 23:47:25.339851 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231410 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231414 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231418 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231422 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231427 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231432 2580 flags.go:64] FLAG: --help="false" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231436 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231440 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231445 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231450 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231455 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231460 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231464 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231468 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231473 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231479 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231484 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231488 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231492 2580 flags.go:64] FLAG: --kube-reserved="" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231497 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231501 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231505 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231509 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231513 2580 flags.go:64] FLAG: --lock-file="" Jul 15 23:47:25.340715 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231517 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231522 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231526 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231533 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231538 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231542 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231546 2580 flags.go:64] FLAG: --logging-format="text" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231550 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231555 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231560 2580 flags.go:64] FLAG: --manifest-url="" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231564 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231570 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231574 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231580 2580 flags.go:64] FLAG: --max-pods="110" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231584 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231588 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231591 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231594 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231597 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231600 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231603 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231610 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231615 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231619 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 23:47:25.345641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231622 2580 flags.go:64] FLAG: --pod-cidr="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231625 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231630 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231633 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231636 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231639 2580 flags.go:64] FLAG: --port="10250" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231642 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231645 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e055bae7843bf829" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231648 2580 flags.go:64] FLAG: --qos-reserved="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231651 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231654 2580 flags.go:64] FLAG: --register-node="true" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231657 2580 flags.go:64] FLAG: --register-with-taints="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231661 2580 flags.go:64] FLAG: --registry-burst="10" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231664 2580 flags.go:64] FLAG: --registry-qps="5" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231667 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231670 2580 flags.go:64] FLAG: --reserved-memory="" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231674 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231676 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231679 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231682 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231685 2580 flags.go:64] FLAG: --runonce="false" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231688 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231691 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231694 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231697 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 23:47:25.544068 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231700 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231702 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231705 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231708 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231711 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231714 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231720 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231723 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231726 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231729 2580 flags.go:64] FLAG: --system-cgroups="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231733 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231738 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231740 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231743 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231747 2580 flags.go:64] FLAG: --tls-min-version="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231750 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231753 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231757 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231759 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231762 2580 flags.go:64] FLAG: --v="2" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231766 2580 flags.go:64] FLAG: --version="false" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231771 2580 flags.go:64] FLAG: --vmodule="" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231774 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.231778 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 23:47:25.567502 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231902 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231906 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231909 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231912 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231915 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231917 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231920 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231923 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231925 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231928 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231930 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231933 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231936 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231938 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231943 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231946 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231948 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231951 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231953 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231956 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.568317 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231959 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231962 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231964 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231967 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231969 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231972 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231974 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231977 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231981 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231985 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231988 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231991 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231994 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.231998 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232002 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232005 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232008 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232010 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232013 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.568910 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232016 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232018 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232021 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232023 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232026 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232029 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232031 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232037 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232039 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232042 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232046 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232049 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232052 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232055 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232058 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232061 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232064 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232066 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232069 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.569542 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232072 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232075 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232077 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232080 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232082 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232085 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232088 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232091 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232094 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232096 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232099 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232102 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232104 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232107 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232109 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232112 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232114 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232117 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232119 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232122 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.570098 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232127 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232130 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232133 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232135 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232138 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232140 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232143 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232146 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232148 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232151 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232154 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232156 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232159 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232161 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232164 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232167 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232169 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232172 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232175 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232177 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.570753 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.232180 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.232750 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.238884 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.238900 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.238918 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.238927 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238981 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238986 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238989 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238992 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238995 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.238998 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239002 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239005 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239007 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.571355 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239010 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239013 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239016 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239019 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239021 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239024 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239027 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239030 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239032 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239034 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239037 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239040 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239043 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239045 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239049 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239051 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239054 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239057 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239060 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239062 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.571845 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239071 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239074 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239076 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239079 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239082 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239084 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239087 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239090 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239092 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239095 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239097 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239100 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239102 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239106 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239110 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239113 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239118 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239121 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239124 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239126 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.572480 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239129 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239131 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239134 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239138 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239140 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239144 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239148 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239151 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239154 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239157 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239160 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239163 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239167 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239177 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239180 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239183 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239186 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239189 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239191 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.573160 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239194 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239197 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239199 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239202 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239204 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239207 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239210 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239212 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239215 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239236 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239240 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239245 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239248 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239250 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239253 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239256 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239258 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239261 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239264 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.573779 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239267 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239269 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239272 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239274 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239277 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239279 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239282 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239284 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239288 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239290 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239293 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239296 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.239301 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239412 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239416 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 23:47:25.574394 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239420 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239423 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239428 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239431 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239434 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239437 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239440 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239442 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239446 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239449 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239451 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239454 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239456 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239459 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239463 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239466 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239468 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239471 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239474 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239476 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 23:47:25.574878 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239479 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239481 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239484 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239486 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239489 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239492 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239495 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239497 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239500 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239503 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239506 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239508 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239511 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239514 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239517 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239520 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239522 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239525 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239528 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239531 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 23:47:25.575517 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239535 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239537 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239540 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239542 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239545 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239547 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239550 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239552 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239555 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239557 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239560 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239562 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239565 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239568 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239571 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239573 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239576 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239578 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239581 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239584 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 23:47:25.576094 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239587 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239589 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239593 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239596 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239599 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239602 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239606 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239608 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239611 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239613 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239616 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239618 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239622 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239625 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239627 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239630 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239632 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239635 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239637 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 23:47:25.576791 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239640 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239642 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239645 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239647 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239650 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239652 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239655 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239657 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239660 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239663 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239665 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239667 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239670 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239673 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239676 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239679 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239683 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 23:47:25.577350 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:25.239686 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.239690 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.239788 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.241835 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.243171 2580 server.go:1019] "Starting client certificate rotation" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.243275 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.243307 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.245715 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.247175 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.263081 2580 log.go:25] "Validated CRI v1 runtime API" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.265131 2580 log.go:25] "Validated CRI v1 image API" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.265921 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 23:47:25.578026 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.270161 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ba3f0bfd-6918-463c-8edd-2f853ac8a7f2:/dev/nvme0n1p4 fe32e065-6bf1-43b6-b27d-40d4c9eb5cdf:/dev/nvme0n1p3] Jul 15 23:47:25.711143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.270183 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.275114 2580 manager.go:216] Machine: {Timestamp:2026-07-15 23:47:25.273728932 +0000 UTC m=+0.329058996 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097154 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2518364d0483f547f8e5c54a05b83b SystemUUID:ec251836-4d04-83f5-47f8-e5c54a05b83b BootID:088303dc-edf0-40fc-96d0-0fabf845fc5c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:90:9d:5c:c0:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:90:9d:5c:c0:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:41:0a:dd:87:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.275258 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.275340 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.277434 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.278266 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.278295 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-211.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 23:47:25.711587 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.279426 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.279437 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.279444 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.279457 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.280265 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.282968 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283089 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283116 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283127 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283150 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283162 2580 kubelet.go:398] "Adding apiserver pod source" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283171 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283473 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.283488 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.285346 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.286682 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.286711 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287285 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287297 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287304 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287310 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287318 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287323 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287336 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287343 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287349 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287354 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287371 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287376 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.287997 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.288003 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.292348 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-211.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.292521 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.292685 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.292856 2580 server.go:1268] "Started kubelet" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.293028 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.293139 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.293178 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.294311 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 23:47:25.713892 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.296533 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.298247 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.298760 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.298814 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.298898 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299442 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299442 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299471 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299573 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299584 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299779 2580 factory.go:55] Registering systemd factory Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299793 2580 factory.go:223] Registration of the systemd container factory successfully Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299988 2580 factory.go:153] Registering CRI-O factory Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.299997 2580 factory.go:223] Registration of the crio container factory successfully Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.300034 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.300054 2580 factory.go:103] Registering Raw factory Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.300065 2580 manager.go:1196] Started watching for new ooms in manager Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.300588 2580 manager.go:349] Starting recovery of all containers Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.298765 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-211.ec2.internal.18c29b6176687b1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-211.ec2.internal,UID:ip-10-0-129-211.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-211.ec2.internal,},FirstTimestamp:2026-07-15 23:47:25.29282947 +0000 UTC m=+0.348159534,LastTimestamp:2026-07-15 23:47:25.29282947 +0000 UTC m=+0.348159534,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-211.ec2.internal,}" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.301840 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8mjqr" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.309005 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.312657 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8mjqr" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.315916 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.315954 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.316304 2580 manager.go:354] Recovery completed Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.317762 2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.320554 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.328054 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.328081 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.328118 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.329687 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.329702 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 23:47:25.715039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.329732 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.329715 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.330652 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.330688 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.330702 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.335778 2580 policy_none.go:49] "None policy: Start" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.335791 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.335801 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.336741 2580 policy_none.go:47] "Start" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.384634 2580 manager.go:337] "Starting Device Plugin manager" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.384778 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.384791 2580 server.go:87] "Starting device plugin registration server" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.385021 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.385034 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.385125 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.385222 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.385232 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.385754 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.385797 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.485578 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.486447 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.486471 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.486482 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.486510 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.494508 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.494524 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-211.ec2.internal\": node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.502424 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.502472 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.502496 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.502539 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.505512 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.521345 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.603663 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal"] Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.603752 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.605149 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.605177 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.605188 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.606849 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.721736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.606959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.606988 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607817 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607826 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607841 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607854 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607854 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.607940 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.609170 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.609191 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.609892 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientMemory" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.609916 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.609929 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeHasSufficientPID" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.621750 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.639982 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-211.ec2.internal\" not found" node="ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.644636 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-211.ec2.internal\" not found" node="ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.700794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.700824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.700847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cd24277ff6091c18d8b9f1eee6e8097-config\") pod \"kube-apiserver-proxy-ip-10-0-129-211.ec2.internal\" (UID: \"5cd24277ff6091c18d8b9f1eee6e8097\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.722865 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.722466 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cd24277ff6091c18d8b9f1eee6e8097-config\") pod \"kube-apiserver-proxy-ip-10-0-129-211.ec2.internal\" (UID: \"5cd24277ff6091c18d8b9f1eee6e8097\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cd24277ff6091c18d8b9f1eee6e8097-config\") pod \"kube-apiserver-proxy-ip-10-0-129-211.ec2.internal\" (UID: \"5cd24277ff6091c18d8b9f1eee6e8097\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801538 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.801559 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1875b2cf2107fb103e1ce838a786f48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal\" (UID: \"e1875b2cf2107fb103e1ce838a786f48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.823927 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:25.924986 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.944578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:25.984913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:25.950562 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:26.224840 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.026078 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.224840 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.127001 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.229123 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.227418 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.242688 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.242811 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.297336 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.318178 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 23:42:25 +0000 UTC" deadline="2027-12-24 11:32:24.340159864 +0000 UTC" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.318208 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12635h44m58.021954436s" Jul 15 23:47:26.428113 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.328005 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.430111 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.429241 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:26.530343 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-211.ec2.internal\" not found" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:26.589017 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1875b2cf2107fb103e1ce838a786f48.slice/crio-b4e5b683151cb794374438201ab86fa17aaebd16b5c6504183d25eb814e93847 WatchSource:0}: Error finding container b4e5b683151cb794374438201ab86fa17aaebd16b5c6504183d25eb814e93847: Status 404 returned error can't find the container with id b4e5b683151cb794374438201ab86fa17aaebd16b5c6504183d25eb814e93847 Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:26.589413 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd24277ff6091c18d8b9f1eee6e8097.slice/crio-665f1d82a19d62992143406b8e74e5b820841929d4cdb9539a78106ef5af4a3f WatchSource:0}: Error finding container 665f1d82a19d62992143406b8e74e5b820841929d4cdb9539a78106ef5af4a3f: Status 404 returned error can't find the container with id 665f1d82a19d62992143406b8e74e5b820841929d4cdb9539a78106ef5af4a3f Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.592899 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.596166 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.599914 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.611260 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.612393 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" Jul 15 23:47:26.675711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.620474 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 23:47:26.928163 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:26.689696 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.277916 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.284145 2580 apiserver.go:52] "Watching apiserver" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.293365 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.293825 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vmwm4","openshift-multus/multus-mssk4","openshift-dns/node-resolver-b96kf","openshift-image-registry/node-ca-2v9r6","openshift-multus/network-metrics-daemon-wrv5h","openshift-network-diagnostics/network-check-target-266qn","openshift-network-operator/iptables-alerter-587sm","openshift-ovn-kubernetes/ovnkube-node-q456c","kube-system/global-pull-secret-syncer-xxp97","kube-system/konnectivity-agent-6l2mk","kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57","openshift-cluster-node-tuning-operator/tuned-wkbmm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal"] Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.295860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.298163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.298697 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.298728 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.299690 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.304472 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.304662 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2gsdh\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.304876 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.305208 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.305577 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.305671 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-862th\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.305941 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.307261 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.308092 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.308808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.308816 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.308834 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.309179 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.309335 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.309443 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8q5mw\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.309546 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ln2q5\"" Jul 15 23:47:27.327111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.309963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.310109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.311745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313784 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3526fa30-56d0-4586-8bca-5cb11fa62eec-host-slash\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysconfig\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-var-lib-kubelet\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3526fa30-56d0-4586-8bca-5cb11fa62eec-iptables-alerter-script\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gprml\" (UniqueName: \"kubernetes.io/projected/3526fa30-56d0-4586-8bca-5cb11fa62eec-kube-api-access-gprml\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.313981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-modprobe-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-kubernetes\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-conf\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-run\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.330689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-host\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zld8w\" (UniqueName: \"kubernetes.io/projected/77d8b75b-5d7b-4ec0-8b92-c48b529da598-kube-api-access-zld8w\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-systemd\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-sys\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314216 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-tmp\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-host\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-serviceca\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2nq\" (UniqueName: \"kubernetes.io/projected/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-kube-api-access-9m2nq\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-lib-modules\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-tuned\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlngm\" (UniqueName: \"kubernetes.io/projected/a193b8ad-87d6-4423-916a-b238ae2dcdcf-kube-api-access-hlngm\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77d8b75b-5d7b-4ec0-8b92-c48b529da598-hosts-file\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d8b75b-5d7b-4ec0-8b92-c48b529da598-tmp-dir\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kc4m\" (UniqueName: \"kubernetes.io/projected/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-kube-api-access-4kc4m\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314585 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314597 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 23:47:27.331621 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314604 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.314917 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.315541 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.315843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.315978 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rnhhl\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.316063 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.316212 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.316337 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.316717 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.316791 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x4p8z\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.317336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.318383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.318749 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-csp52\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.319509 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.320625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.320691 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.322769 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.322949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.324524 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 23:47:27.335521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.324736 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 23:47:27.341256 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.339666 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 23:47:27.341256 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.323100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9h5mn\"" Jul 15 23:47:27.343608 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.341780 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f8pg6" Jul 15 23:47:27.343608 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.343401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.428631 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.345006 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f8pg6" Jul 15 23:47:27.428631 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.348485 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 23:47:27.428631 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.348606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4q4rg\"" Jul 15 23:47:27.428631 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.348672 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 23:47:27.438217 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77d8b75b-5d7b-4ec0-8b92-c48b529da598-hosts-file\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.438217 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d8b75b-5d7b-4ec0-8b92-c48b529da598-tmp-dir\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.438217 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kc4m\" (UniqueName: \"kubernetes.io/projected/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-kube-api-access-4kc4m\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.438217 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3526fa30-56d0-4586-8bca-5cb11fa62eec-host-slash\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.438217 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysconfig\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-var-lib-kubelet\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3526fa30-56d0-4586-8bca-5cb11fa62eec-iptables-alerter-script\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gprml\" (UniqueName: \"kubernetes.io/projected/3526fa30-56d0-4586-8bca-5cb11fa62eec-kube-api-access-gprml\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-modprobe-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-kubernetes\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-conf\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.438633 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-run\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.439182 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-run\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.439182 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.438888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77d8b75b-5d7b-4ec0-8b92-c48b529da598-hosts-file\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.439374 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.439184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77d8b75b-5d7b-4ec0-8b92-c48b529da598-tmp-dir\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.439385 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3526fa30-56d0-4586-8bca-5cb11fa62eec-host-slash\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.439462 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysconfig\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.439658 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.439732 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-var-lib-kubelet\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.439982 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.440098 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:27.940044651 +0000 UTC m=+2.995374715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.440510 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-kubernetes\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.440821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-modprobe-d\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.440831 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-sysctl-conf\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.441083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-host\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.441772 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3526fa30-56d0-4586-8bca-5cb11fa62eec-iptables-alerter-script\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.446394 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.451710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gprml\" (UniqueName: \"kubernetes.io/projected/3526fa30-56d0-4586-8bca-5cb11fa62eec-kube-api-access-gprml\") pod \"iptables-alerter-587sm\" (UID: \"3526fa30-56d0-4586-8bca-5cb11fa62eec\") " pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.454478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kc4m\" (UniqueName: \"kubernetes.io/projected/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-kube-api-access-4kc4m\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.532342 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zld8w\" (UniqueName: \"kubernetes.io/projected/77d8b75b-5d7b-4ec0-8b92-c48b529da598-kube-api-access-zld8w\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-systemd\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-sys\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467932 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-sys\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467956 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-host\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-tmp\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.467995 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-systemd\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.468024 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-host\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.468079 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-host\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.468135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-serviceca\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.468216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2nq\" (UniqueName: \"kubernetes.io/projected/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-kube-api-access-9m2nq\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.468480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-lib-modules\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.469792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-tuned\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.469822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlngm\" (UniqueName: \"kubernetes.io/projected/a193b8ad-87d6-4423-916a-b238ae2dcdcf-kube-api-access-hlngm\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.469893 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.469755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-serviceca\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.533143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.469001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a193b8ad-87d6-4423-916a-b238ae2dcdcf-lib-modules\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.471341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-tmp\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.473151 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a193b8ad-87d6-4423-916a-b238ae2dcdcf-etc-tuned\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.477631 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zld8w\" (UniqueName: \"kubernetes.io/projected/77d8b75b-5d7b-4ec0-8b92-c48b529da598-kube-api-access-zld8w\") pod \"node-resolver-b96kf\" (UID: \"77d8b75b-5d7b-4ec0-8b92-c48b529da598\") " pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.478324 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlngm\" (UniqueName: \"kubernetes.io/projected/a193b8ad-87d6-4423-916a-b238ae2dcdcf-kube-api-access-hlngm\") pod \"tuned-wkbmm\" (UID: \"a193b8ad-87d6-4423-916a-b238ae2dcdcf\") " pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.484163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2nq\" (UniqueName: \"kubernetes.io/projected/b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4-kube-api-access-9m2nq\") pod \"node-ca-2v9r6\" (UID: \"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4\") " pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.508288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" event={"ID":"e1875b2cf2107fb103e1ce838a786f48","Type":"ContainerStarted","Data":"b4e5b683151cb794374438201ab86fa17aaebd16b5c6504183d25eb814e93847"} Jul 15 23:47:27.534442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.524483 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" event={"ID":"5cd24277ff6091c18d8b9f1eee6e8097","Type":"ContainerStarted","Data":"665f1d82a19d62992143406b8e74e5b820841929d4cdb9539a78106ef5af4a3f"} Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-bin\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-netd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-env-overrides\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-multus\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmtt8\" (UniqueName: \"kubernetes.io/projected/35d22e55-319e-442c-801d-16be197f1831-kube-api-access-kmtt8\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-etc-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-node-log\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-socket-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.570965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-os-release\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.636731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.571064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-slash\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.571120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-netns\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cni-binary-copy\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-netns\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nlzd\" (UniqueName: \"kubernetes.io/projected/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-kube-api-access-5nlzd\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-device-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.578377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-os-release\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns6xp\" (UniqueName: \"kubernetes.io/projected/5b15672f-ce5c-470f-bfd0-c756d4d3f910-kube-api-access-ns6xp\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-dbus\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-systemd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-script-lib\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-registration-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-k8s-cni-cncf-io\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.637418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-daemon-config\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-multus-certs\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-log-socket\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-bin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-kubelet\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-etc-kubernetes\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-agent-certs\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-hostroot\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-kubelet-config\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-systemd-units\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9bjp\" (UniqueName: \"kubernetes.io/projected/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-kube-api-access-g9bjp\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.579997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.638787 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-system-cni-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-var-lib-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-sys-fs\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-config\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovn-node-metrics-cert\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-konnectivity-ca\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cnibin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-kubelet\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-system-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-socket-dir-parent\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.639468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-conf-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.640620 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.640620 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cnibin\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.640620 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:27.640620 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.580599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-ovn\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.641761 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.640982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" Jul 15 23:47:27.641761 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.641077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-587sm" Jul 15 23:47:27.651875 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.651840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2v9r6" Jul 15 23:47:27.654630 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:27.654608 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda193b8ad_87d6_4423_916a_b238ae2dcdcf.slice/crio-1193be2b423ec5249f6feee84c2d425e7dda5bab23b9c72847bf95f7881410d3 WatchSource:0}: Error finding container 1193be2b423ec5249f6feee84c2d425e7dda5bab23b9c72847bf95f7881410d3: Status 404 returned error can't find the container with id 1193be2b423ec5249f6feee84c2d425e7dda5bab23b9c72847bf95f7881410d3 Jul 15 23:47:27.655265 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:27.655239 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3526fa30_56d0_4586_8bca_5cb11fa62eec.slice/crio-8270586041c62c8b75d346c2f21c8dd3bd8053dbbe0f303acc37b5bf1a7e1d39 WatchSource:0}: Error finding container 8270586041c62c8b75d346c2f21c8dd3bd8053dbbe0f303acc37b5bf1a7e1d39: Status 404 returned error can't find the container with id 8270586041c62c8b75d346c2f21c8dd3bd8053dbbe0f303acc37b5bf1a7e1d39 Jul 15 23:47:27.657158 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.657126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b96kf" Jul 15 23:47:27.663855 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:27.663830 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7fcbfe1_67e5_4f4f_bfec_e8577a815dd4.slice/crio-d6e33626ff54dc34694d945d3bf4f036ceec52119724e81055ef92cdb351f311 WatchSource:0}: Error finding container d6e33626ff54dc34694d945d3bf4f036ceec52119724e81055ef92cdb351f311: Status 404 returned error can't find the container with id d6e33626ff54dc34694d945d3bf4f036ceec52119724e81055ef92cdb351f311 Jul 15 23:47:27.666285 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:27.666240 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d8b75b_5d7b_4ec0_8b92_c48b529da598.slice/crio-27bcb55dbc3ef717a2afd99d236493a52ffd0cc52da3a132ad4e6f0d10c19731 WatchSource:0}: Error finding container 27bcb55dbc3ef717a2afd99d236493a52ffd0cc52da3a132ad4e6f0d10c19731: Status 404 returned error can't find the container with id 27bcb55dbc3ef717a2afd99d236493a52ffd0cc52da3a132ad4e6f0d10c19731 Jul 15 23:47:27.681700 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.681817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-socket-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.681817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.681817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-os-release\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.681817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-slash\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-netns\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cni-binary-copy\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-os-release\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-netns\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nlzd\" (UniqueName: \"kubernetes.io/projected/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-kube-api-access-5nlzd\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681929 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-netns\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-device-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-os-release\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.682027 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:27.682033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682030 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-netns\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.682077 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:28.182059046 +0000 UTC m=+3.237389099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682077 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-device-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682152 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-os-release\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.681996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-socket-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-slash\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6xp\" (UniqueName: \"kubernetes.io/projected/5b15672f-ce5c-470f-bfd0-c756d4d3f910-kube-api-access-ns6xp\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-dbus\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-systemd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-script-lib\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-registration-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-k8s-cni-cncf-io\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-daemon-config\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-multus-certs\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.682755 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-log-socket\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-bin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-kubelet\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682532 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-etc-kubernetes\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-agent-certs\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-hostroot\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-kubelet-config\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-systemd-units\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9bjp\" (UniqueName: \"kubernetes.io/projected/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-kube-api-access-g9bjp\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cni-binary-copy\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-system-cni-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-var-lib-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.683569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-etc-kubernetes\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-sys-fs\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682874 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-registration-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-config\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-k8s-cni-cncf-io\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovn-node-metrics-cert\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.682985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-konnectivity-ca\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683024 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-script-lib\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-dbus\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cnibin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-kubelet\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.684326 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-system-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-socket-dir-parent\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-conf-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cnibin\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovnkube-config\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-ovn\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-bin\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-netd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-env-overrides\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-multus\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmtt8\" (UniqueName: \"kubernetes.io/projected/35d22e55-319e-442c-801d-16be197f1831-kube-api-access-kmtt8\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.685037 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-etc-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-node-log\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683575 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-daemon-config\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-systemd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683641 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-node-log\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-cnibin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683694 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-conf-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-socket-dir-parent\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683732 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-run-ovn-kubernetes\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683793 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-kubelet\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683827 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-system-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-hostroot\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684157 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ab8a1c8-57e8-47b4-999b-63f569069d7a-kubelet-config\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-systemd-units\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.685646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-konnectivity-ca\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684327 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-log-socket\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684359 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684379 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-bin\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-system-cni-dir\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684429 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-kubelet\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684420 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-multus-cni-dir\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684458 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-var-lib-cni-multus\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35d22e55-319e-442c-801d-16be197f1831-sys-fs\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684526 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-run-ovn\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cnibin\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684589 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-var-lib-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-etc-openvswitch\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.683696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-host-run-multus-certs\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.686651 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684685 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-bin\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686651 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.684705 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-host-cni-netd\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.686651 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.685043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-binary-copy\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.686651 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.685080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b15672f-ce5c-470f-bfd0-c756d4d3f910-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.686651 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.685338 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-env-overrides\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.687056 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.687035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eebb966-f5ca-42e6-8add-3fc2f7d25a2e-agent-certs\") pod \"konnectivity-agent-6l2mk\" (UID: \"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e\") " pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.687313 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.687276 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-ovn-node-metrics-cert\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.690601 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.690500 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:27.690601 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.690522 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:27.690601 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.690535 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:27.690601 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.690605 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:28.190584356 +0000 UTC m=+3.245914431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:27.691336 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.691312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6xp\" (UniqueName: \"kubernetes.io/projected/5b15672f-ce5c-470f-bfd0-c756d4d3f910-kube-api-access-ns6xp\") pod \"multus-additional-cni-plugins-vmwm4\" (UID: \"5b15672f-ce5c-470f-bfd0-c756d4d3f910\") " pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.692275 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.692244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nlzd\" (UniqueName: \"kubernetes.io/projected/daefb2df-f3f8-482e-8bf8-f180c5bbf3b8-kube-api-access-5nlzd\") pod \"multus-mssk4\" (UID: \"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8\") " pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.692388 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.692366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:27.693541 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.693518 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmtt8\" (UniqueName: \"kubernetes.io/projected/35d22e55-319e-442c-801d-16be197f1831-kube-api-access-kmtt8\") pod \"aws-ebs-csi-driver-node-cjf57\" (UID: \"35d22e55-319e-442c-801d-16be197f1831\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.693774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.693724 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9bjp\" (UniqueName: \"kubernetes.io/projected/df3bfbc9-4d64-4eef-a8ec-5f5fed922438-kube-api-access-g9bjp\") pod \"ovnkube-node-q456c\" (UID: \"df3bfbc9-4d64-4eef-a8ec-5f5fed922438\") " pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.706618 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:27.706588 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eebb966_f5ca_42e6_8add_3fc2f7d25a2e.slice/crio-061cf71f6541ce33641f77802a646378f3a2a84f51f0335277e2a33e2a7300ad WatchSource:0}: Error finding container 061cf71f6541ce33641f77802a646378f3a2a84f51f0335277e2a33e2a7300ad: Status 404 returned error can't find the container with id 061cf71f6541ce33641f77802a646378f3a2a84f51f0335277e2a33e2a7300ad Jul 15 23:47:27.746570 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.746542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" Jul 15 23:47:27.966183 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.966092 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:27.973241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.973201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" Jul 15 23:47:27.985827 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.985800 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mssk4" Jul 15 23:47:27.985951 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:27.985934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:27.986069 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.986052 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:27.986145 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:27.986124 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:28.986103268 +0000 UTC m=+4.041433324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:28.188200 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.187578 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:28.188200 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.187769 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:28.188200 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.187834 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:29.187813507 +0000 UTC m=+4.243143567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:28.289413 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.288738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:28.289413 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.288940 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:28.289413 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.288962 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:28.289413 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.288975 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:28.289413 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.289031 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:29.28901139 +0000 UTC m=+4.344341465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:28.347970 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.347910 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 23:42:27 +0000 UTC" deadline="2028-02-11 13:14:59.888728151 +0000 UTC" Jul 15 23:47:28.347970 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.347969 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13813h27m31.540763681s" Jul 15 23:47:28.503132 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.503106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:28.504000 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.503256 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:28.529812 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.529778 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2v9r6" event={"ID":"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4","Type":"ContainerStarted","Data":"d6e33626ff54dc34694d945d3bf4f036ceec52119724e81055ef92cdb351f311"} Jul 15 23:47:28.533989 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.533908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-587sm" event={"ID":"3526fa30-56d0-4586-8bca-5cb11fa62eec","Type":"ContainerStarted","Data":"8270586041c62c8b75d346c2f21c8dd3bd8053dbbe0f303acc37b5bf1a7e1d39"} Jul 15 23:47:28.539175 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.539119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" event={"ID":"a193b8ad-87d6-4423-916a-b238ae2dcdcf","Type":"ContainerStarted","Data":"1193be2b423ec5249f6feee84c2d425e7dda5bab23b9c72847bf95f7881410d3"} Jul 15 23:47:28.542875 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.542798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l2mk" event={"ID":"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e","Type":"ContainerStarted","Data":"061cf71f6541ce33641f77802a646378f3a2a84f51f0335277e2a33e2a7300ad"} Jul 15 23:47:28.544668 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.544600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b96kf" event={"ID":"77d8b75b-5d7b-4ec0-8b92-c48b529da598","Type":"ContainerStarted","Data":"27bcb55dbc3ef717a2afd99d236493a52ffd0cc52da3a132ad4e6f0d10c19731"} Jul 15 23:47:28.994607 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:28.994570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:28.994787 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.994763 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:28.994851 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:28.994842 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:30.994820862 +0000 UTC m=+6.050150934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:29.196365 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.196272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:29.196539 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.196475 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:29.196609 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.196554 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:31.196534121 +0000 UTC m=+6.251864193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:29.297428 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.297381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:29.297612 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.297544 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:29.297612 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.297564 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:29.297612 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.297576 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:29.297838 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.297634 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:31.297615817 +0000 UTC m=+6.352945870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:29.349209 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.349136 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 23:42:27 +0000 UTC" deadline="2027-12-24 14:56:56.708069553 +0000 UTC" Jul 15 23:47:29.349209 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.349179 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12639h9m27.358894818s" Jul 15 23:47:29.509496 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.508801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:29.509496 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:29.508812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:29.509496 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.508932 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:29.509496 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:29.509405 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:30.183833 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:30.183797 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d22e55_319e_442c_801d_16be197f1831.slice/crio-63d29a065f3ac216d21343ccb06e84660924877c6478157819f9d0c0b4d8d06e WatchSource:0}: Error finding container 63d29a065f3ac216d21343ccb06e84660924877c6478157819f9d0c0b4d8d06e: Status 404 returned error can't find the container with id 63d29a065f3ac216d21343ccb06e84660924877c6478157819f9d0c0b4d8d06e Jul 15 23:47:30.199647 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:47:30.199609 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaefb2df_f3f8_482e_8bf8_f180c5bbf3b8.slice/crio-18de8760a0d3d08352936b9f68fb0a7662efd0d959ec36710ea41a63997f18e8 WatchSource:0}: Error finding container 18de8760a0d3d08352936b9f68fb0a7662efd0d959ec36710ea41a63997f18e8: Status 404 returned error can't find the container with id 18de8760a0d3d08352936b9f68fb0a7662efd0d959ec36710ea41a63997f18e8 Jul 15 23:47:30.504184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:30.503688 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:30.504184 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:30.503829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:30.550974 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:30.550850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"259671316bd405c4ac1817da7f0481f58a7f00bb529e21ea22f94d5e5e8f2501"} Jul 15 23:47:30.552411 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:30.552370 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mssk4" event={"ID":"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8","Type":"ContainerStarted","Data":"18de8760a0d3d08352936b9f68fb0a7662efd0d959ec36710ea41a63997f18e8"} Jul 15 23:47:30.553710 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:30.553685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" event={"ID":"35d22e55-319e-442c-801d-16be197f1831","Type":"ContainerStarted","Data":"63d29a065f3ac216d21343ccb06e84660924877c6478157819f9d0c0b4d8d06e"} Jul 15 23:47:31.011124 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.010748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:31.011124 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.010950 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:31.011124 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.011019 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:35.011000333 +0000 UTC m=+10.066330389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:31.213299 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.213257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:31.213465 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.213445 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:31.213542 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.213504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:35.213485386 +0000 UTC m=+10.268815438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:31.313926 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.313798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:31.314070 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.313945 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:31.314070 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.313963 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:31.314070 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.313974 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:31.314070 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.314032 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:35.314015109 +0000 UTC m=+10.369345174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:31.503663 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.503634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:31.503826 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.503634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:31.503826 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.503768 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:31.503826 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:31.503814 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:31.556832 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:31.556803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerStarted","Data":"a3d2d0a3ec26f1d50cb41aec5bda0ac429da86ffcf3590e907a54080e8417e85"} Jul 15 23:47:32.503465 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:32.503433 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:32.503658 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:32.503559 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:33.503578 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:33.503507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:33.503980 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:33.503653 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:33.503980 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:33.503698 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:33.503980 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:33.503811 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:34.503254 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:34.503196 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:34.503455 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:34.503362 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:35.045297 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:35.045256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:35.045739 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.045452 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:35.045739 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.045516 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:43.045495129 +0000 UTC m=+18.100825186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:35.248254 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:35.247973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:35.248254 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.248160 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:35.248831 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.248810 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:43.248200139 +0000 UTC m=+18.303530199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:35.349237 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:35.349135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:35.349423 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.349345 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:35.349423 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.349366 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:35.349423 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.349378 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:35.349647 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.349435 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:43.349418493 +0000 UTC m=+18.404748566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:35.504201 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:35.504162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:35.504383 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.504302 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:35.504685 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:35.504501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:35.504685 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:35.504612 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:36.503164 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:36.503132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:36.503614 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:36.503282 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:37.503286 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:37.503252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:37.503736 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:37.503397 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:37.503832 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:37.503805 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:37.503966 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:37.503911 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:38.503512 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:38.503488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:38.503973 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:38.503611 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:38.573482 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:38.573448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" event={"ID":"a193b8ad-87d6-4423-916a-b238ae2dcdcf","Type":"ContainerStarted","Data":"73ffcedf237969dd32b8690383daafbd5f966b4fc9092fb1c62a3da0bffdef95"} Jul 15 23:47:38.575722 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:38.575693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" event={"ID":"5cd24277ff6091c18d8b9f1eee6e8097","Type":"ContainerStarted","Data":"3513c7e24d2b75d4ccf8b26217e57063468a4c1595b937e63876c936760e6f50"} Jul 15 23:47:38.609321 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:38.609259 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wkbmm" podStartSLOduration=3.577141057 podStartE2EDuration="13.609207796s" totalImagesPullingTime="10.032066739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:38.596283616 +0000 UTC m=+13.651613689" watchObservedRunningTime="2026-07-15 23:47:38.609207796 +0000 UTC m=+13.664537870" Jul 15 23:47:38.609963 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:38.609914 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-211.ec2.internal" podStartSLOduration=12.60990492 podStartE2EDuration="12.60990492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:38.60915539 +0000 UTC m=+13.664485464" watchObservedRunningTime="2026-07-15 23:47:38.60990492 +0000 UTC m=+13.665234995" Jul 15 23:47:39.504022 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.503774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:39.504421 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.503838 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:39.504421 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:39.504078 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:39.504421 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:39.504155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:39.579409 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.579373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" event={"ID":"e1875b2cf2107fb103e1ce838a786f48","Type":"ContainerStarted","Data":"0541f58adf326a04ed28048a5b760547d086cf4acd5d62767e63b796713f2130"} Jul 15 23:47:39.581091 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.581061 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l2mk" event={"ID":"7eebb966-f5ca-42e6-8add-3fc2f7d25a2e","Type":"ContainerStarted","Data":"c390e9ec01fe5d0d4816d01e0cbff17f18910e753a7b53cf900702649781d58e"} Jul 15 23:47:39.582673 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.582643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b96kf" event={"ID":"77d8b75b-5d7b-4ec0-8b92-c48b529da598","Type":"ContainerStarted","Data":"b3030d1b6a07565f1fa73ace621fad57158d0d44a5d45aa43aacee5036d0dddd"} Jul 15 23:47:39.584147 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.584127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2v9r6" event={"ID":"b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4","Type":"ContainerStarted","Data":"73a2a82967b2575b23380fe47a49241400902446536c33bcfd73e9dda5f561eb"} Jul 15 23:47:39.585619 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.585589 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-587sm" event={"ID":"3526fa30-56d0-4586-8bca-5cb11fa62eec","Type":"ContainerStarted","Data":"41abcb108a5f213aba5cf79c3138bf0140554788b512e7d7629730af049d84f6"} Jul 15 23:47:39.587059 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.587035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerStarted","Data":"eae2105032a556f22a247e88316b3c7d84de4f965c18f0bcf5edcdd2d9a04841"} Jul 15 23:47:39.588583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.588561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" event={"ID":"35d22e55-319e-442c-801d-16be197f1831","Type":"ContainerStarted","Data":"af6f48224555204b7a35db7b2ca99e4c05cb44daedf086ebc3d0e3b7393c4970"} Jul 15 23:47:39.612798 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.612761 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2v9r6" podStartSLOduration=6.392192859 podStartE2EDuration="14.612747903s" totalImagesPullingTime="8.220555044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:39.612175286 +0000 UTC m=+14.667505360" watchObservedRunningTime="2026-07-15 23:47:39.612747903 +0000 UTC m=+14.668077975" Jul 15 23:47:39.626561 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.626521 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b96kf" podStartSLOduration=4.607756037 podStartE2EDuration="14.626503372s" totalImagesPullingTime="10.018747335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:39.626354426 +0000 UTC m=+14.681684498" watchObservedRunningTime="2026-07-15 23:47:39.626503372 +0000 UTC m=+14.681833446" Jul 15 23:47:39.659465 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.659418 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6l2mk" podStartSLOduration=5.583477958 podStartE2EDuration="14.659401151s" totalImagesPullingTime="9.075923193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:39.639886639 +0000 UTC m=+14.695216711" watchObservedRunningTime="2026-07-15 23:47:39.659401151 +0000 UTC m=+14.714731226" Jul 15 23:47:39.673092 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.673051 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-587sm" podStartSLOduration=5.546886808 podStartE2EDuration="14.6730392s" totalImagesPullingTime="9.126152392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:39.672868279 +0000 UTC m=+14.728198356" watchObservedRunningTime="2026-07-15 23:47:39.6730392 +0000 UTC m=+14.728369274" Jul 15 23:47:39.879653 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.879569 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:39.880287 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:39.880267 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:40.138744 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:40.138657 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:40.256537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:40.256507 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:40.503315 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:40.503243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:40.503451 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:40.503352 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:40.829579 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:40.829499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:41.005483 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.005447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:41.407653 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.407618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:41.502737 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.502710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:41.502896 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:41.502839 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:41.503004 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.502983 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:41.503113 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:41.503092 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:41.592328 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.592304 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 23:47:41.605605 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.605573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:41.654793 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.654768 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:41.816263 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:41.816168 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:42.198353 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:42.198276 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:42.503446 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:42.503377 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:42.503616 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:42.503494 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:42.596733 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:42.596703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:42.800642 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:42.800565 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:42.866398 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:42.866363 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:43.108747 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.108665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:43.108888 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.108827 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:43.108937 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.108897 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:59.108881141 +0000 UTC m=+34.164211193 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:43.310349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.310301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:43.310917 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.310478 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:43.310917 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.310556 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:47:59.31053786 +0000 UTC m=+34.365867910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:43.401531 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.401426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:43.411493 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.411456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:43.411639 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.411594 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:43.411639 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.411612 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:43.411639 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.411621 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:43.411803 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.411672 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:47:59.4116589 +0000 UTC m=+34.466988952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:43.502838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.502797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:43.503003 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.502946 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:43.503060 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.502999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:43.503127 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:43.503104 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:43.597754 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.597712 2580 generic.go:356] "Generic (PLEG): container finished" podID="e1875b2cf2107fb103e1ce838a786f48" containerID="0541f58adf326a04ed28048a5b760547d086cf4acd5d62767e63b796713f2130" exitCode=0 Jul 15 23:47:43.597915 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.597747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" event={"ID":"e1875b2cf2107fb103e1ce838a786f48","Type":"ContainerDied","Data":"0541f58adf326a04ed28048a5b760547d086cf4acd5d62767e63b796713f2130"} Jul 15 23:47:43.598302 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.598274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:43.599754 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.599726 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="eae2105032a556f22a247e88316b3c7d84de4f965c18f0bcf5edcdd2d9a04841" exitCode=0 Jul 15 23:47:43.599849 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.599770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"eae2105032a556f22a247e88316b3c7d84de4f965c18f0bcf5edcdd2d9a04841"} Jul 15 23:47:43.650490 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.650452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:43.794872 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:43.794772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:44.001274 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:44.001239 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:44.123901 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:44.123807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:44.503695 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:44.503619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:44.504138 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:44.503732 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:45.199488 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.199452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:45.397005 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.396978 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:45.504104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.504035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:45.504104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.504078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:45.504549 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:45.504156 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:45.504549 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:45.504277 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:45.603786 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.603753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:45.802329 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:45.802251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:46.005151 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:46.005120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:46.062887 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:46.062817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:46.503072 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:46.503010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:46.503192 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:46.503110 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:47.001509 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.001484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:47.225744 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.225665 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 23:47:47.394748 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.394566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:47.396799 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.396691 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T23:47:47.225686726Z","UUID":"6765525d-ecf5-40ce-8af8-ab3544637535","Handler":null,"Name":"","Endpoint":""} Jul 15 23:47:47.398716 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.398680 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 23:47:47.398716 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.398715 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 23:47:47.502885 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.502850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:47.503046 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:47.503003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:47.503255 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.502859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:47.503369 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:47.503345 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:47.599913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.599869 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:47.608397 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.608364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" event={"ID":"e1875b2cf2107fb103e1ce838a786f48","Type":"ContainerStarted","Data":"07b6d9cbb94f9754f33100bb2c0bade8ea2c1abeb38fecd09c9be41d6e5ee10b"} Jul 15 23:47:47.611687 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"b472b3d7fdd0cf4bab630b7caa5258a0522a7d19d3e8ae8c26fdae3e49f73751"} Jul 15 23:47:47.611788 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611715 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"52476cfd998f0575e44b5bfe3ce7106c30db3349ef5eac4367ee23c6ca83213d"} Jul 15 23:47:47.611849 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611768 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"0600615a8f6aae4104df705a069fae81b5edd37d88b48a0d582c63d42bc03db8"} Jul 15 23:47:47.611903 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"608c0793b01d870d02339849a84d29db2912f4092a6f4bd8653323f81cebe680"} Jul 15 23:47:47.611954 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"888dfc74107b7950fa82d00d21ca82c35a208a88118ff643f0f6c36632a5a583"} Jul 15 23:47:47.612024 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.611975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"8a305172f577c0ad8966013b77d46d66ebb4e97a6977397bbdc5171b86a34b63"} Jul 15 23:47:47.613232 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.613199 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mssk4" event={"ID":"daefb2df-f3f8-482e-8bf8-f180c5bbf3b8","Type":"ContainerStarted","Data":"f4b93c0825253794af77f62e184479414d23885e47c8708b18c1a80a1a57c741"} Jul 15 23:47:47.615013 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.614963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" event={"ID":"35d22e55-319e-442c-801d-16be197f1831","Type":"ContainerStarted","Data":"462f5ed562d56d3c47bf3b9a0713c5e6b95535c5ccb6090c955a213a76de0c78"} Jul 15 23:47:47.623999 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.623955 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-211.ec2.internal" podStartSLOduration=21.623943376 podStartE2EDuration="21.623943376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:47.623659589 +0000 UTC m=+22.678989663" watchObservedRunningTime="2026-07-15 23:47:47.623943376 +0000 UTC m=+22.679273450" Jul 15 23:47:47.639554 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.639513 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mssk4" podStartSLOduration=5.65767539 podStartE2EDuration="22.639501519s" totalImagesPullingTime="16.981826129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:47.639235421 +0000 UTC m=+22.694565495" watchObservedRunningTime="2026-07-15 23:47:47.639501519 +0000 UTC m=+22.694831592" Jul 15 23:47:47.797104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.797068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:47.998980 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:47.998857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:48.503181 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:48.503158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:48.503557 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:48.503295 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:48.619759 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:48.619731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" event={"ID":"35d22e55-319e-442c-801d-16be197f1831","Type":"ContainerStarted","Data":"bf33a738f337dcffb095542e56e9d5c1c0e9572a9f6e2145c25299c2489dfa7a"} Jul 15 23:47:48.637338 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:48.637297 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjf57" podStartSLOduration=6.655206768 podStartE2EDuration="23.637282604s" totalImagesPullingTime="16.982075836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:48.636833 +0000 UTC m=+23.692163094" watchObservedRunningTime="2026-07-15 23:47:48.637282604 +0000 UTC m=+23.692612677" Jul 15 23:47:49.002689 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.002659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.103754 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.103717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.155821 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.155782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.224200 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.224128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.280359 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.280282 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.340317 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.340277 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.405799 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.405763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.502914 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.502889 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:49.503056 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:49.503023 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:49.503125 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.503081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:49.503208 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:49.503190 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:49.517173 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.517147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.617355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.617284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.717193 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.717154 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:49.817562 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:49.817525 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:50.096377 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.096296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:50.288143 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.288107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:50.342396 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.342368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:50.503179 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.503145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:50.503343 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:50.503298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:50.627045 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.626967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"c99c32047bfe3782e0633e7afa1755d96e2b2eb9c70c58ddf0c8bb1052588b75"} Jul 15 23:47:50.892478 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:50.892423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:51.097517 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.097492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:51.352687 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.352639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:51.502872 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.502856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:51.502961 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:51.502946 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:51.503043 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.503026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:51.503125 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:51.503107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:51.512065 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.512038 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:51.613695 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.613627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:51.630773 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.630748 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="2a7ddd0a0942e77dabfe20201aa81430f76b30767d906c0ec750f5634cee1da7" exitCode=0 Jul 15 23:47:51.630866 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:51.630790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"2a7ddd0a0942e77dabfe20201aa81430f76b30767d906c0ec750f5634cee1da7"} Jul 15 23:47:52.503273 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.503253 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:52.503357 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:52.503341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:52.635140 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.635011 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="e595b1dc5747a5954dee13e414d0365a5b6d9a01127db3f09a8b27df4fdfd2e8" exitCode=0 Jul 15 23:47:52.635581 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.635087 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"e595b1dc5747a5954dee13e414d0365a5b6d9a01127db3f09a8b27df4fdfd2e8"} Jul 15 23:47:52.639190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.639163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" event={"ID":"df3bfbc9-4d64-4eef-a8ec-5f5fed922438","Type":"ContainerStarted","Data":"1bfc11d2434320c8124918623d3e1ed1000a900cc9f83e389be4b5e5cf834d13"} Jul 15 23:47:52.639415 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.639399 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:52.654685 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.654600 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:52.682269 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:52.682208 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" podStartSLOduration=10.40155172 podStartE2EDuration="27.68219459s" totalImagesPullingTime="17.28064287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:47:52.680692259 +0000 UTC m=+27.736022332" watchObservedRunningTime="2026-07-15 23:47:52.68219459 +0000 UTC m=+27.737524663" Jul 15 23:47:53.360656 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.360632 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:53.360746 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.360738 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 23:47:53.364336 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.363356 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6l2mk" Jul 15 23:47:53.503733 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.503708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:53.503853 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:53.503831 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:53.503927 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.503910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:53.504019 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:53.504002 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:53.643155 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.643107 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="26900416365aa2d0ecc67c351bb8a2a682d87c26cf2e06ca3871317088ede9f3" exitCode=0 Jul 15 23:47:53.643461 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.643176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"26900416365aa2d0ecc67c351bb8a2a682d87c26cf2e06ca3871317088ede9f3"} Jul 15 23:47:53.643647 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.643634 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:53.643707 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.643697 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:53.658168 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:53.658151 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:47:54.502927 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:54.502887 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:54.503090 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:54.502997 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:55.503995 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:55.503951 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:55.504449 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:55.504113 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:55.504449 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:55.504160 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:55.504449 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:55.504287 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:56.503205 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:56.503176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:56.503397 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:56.503304 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:56.521456 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:56.521430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:56.596970 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:56.596929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:56.818304 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:56.818182 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:56.915266 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:56.915197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:57.002034 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:57.002003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:57.201671 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:57.201595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:57.275333 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:57.275298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:57.503577 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:57.503552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:57.503713 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:57.503679 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:57.503768 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:57.503735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:57.503857 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:57.503820 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:58.465947 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.465752 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-266qn"] Jul 15 23:47:58.466336 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.466035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:58.466336 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:58.466148 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:47:58.468925 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.468898 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxp97"] Jul 15 23:47:58.469052 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.468990 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:58.469110 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:58.469084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:47:58.469580 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.469557 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wrv5h"] Jul 15 23:47:58.469691 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.469653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:58.469792 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:58.469769 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:58.569528 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.569470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:58.628004 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.627967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:58.660149 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.660122 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="a507978779eb181c955b9be58754c878ce424efd23ecf49380431766dc4f4db6" exitCode=0 Jul 15 23:47:58.660265 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.660169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"a507978779eb181c955b9be58754c878ce424efd23ecf49380431766dc4f4db6"} Jul 15 23:47:58.718773 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:58.718751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:59.128652 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.128627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:59.128946 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.128764 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:59.128946 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.128823 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs podName:7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9 nodeName:}" failed. No retries permitted until 2026-07-15 23:48:31.128809281 +0000 UTC m=+66.184139333 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs") pod "network-metrics-daemon-wrv5h" (UID: "7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 23:47:59.329477 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.329412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:47:59.329591 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.329518 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:59.329591 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.329569 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret podName:6ab8a1c8-57e8-47b4-999b-63f569069d7a nodeName:}" failed. No retries permitted until 2026-07-15 23:48:31.329551704 +0000 UTC m=+66.384881774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret") pod "global-pull-secret-syncer-xxp97" (UID: "6ab8a1c8-57e8-47b4-999b-63f569069d7a") : object "kube-system"/"original-pull-secret" not registered Jul 15 23:47:59.430204 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.430180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:47:59.430322 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.430302 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 23:47:59.430322 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.430316 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 23:47:59.430394 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.430324 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tn6ws for pod openshift-network-diagnostics/network-check-target-266qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:59.430394 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.430365 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws podName:5e664898-1dcd-4d36-9a2e-9fcc70c79725 nodeName:}" failed. No retries permitted until 2026-07-15 23:48:31.430351783 +0000 UTC m=+66.485681834 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tn6ws" (UniqueName: "kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws") pod "network-check-target-266qn" (UID: "5e664898-1dcd-4d36-9a2e-9fcc70c79725") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 23:47:59.503054 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.503035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:47:59.503560 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:47:59.503122 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:47:59.665010 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.664947 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b15672f-ce5c-470f-bfd0-c756d4d3f910" containerID="c1b4ed2c83cf538a86cd0a4173bbb0dfeb6fbe52bef94f837b25edbba62d336c" exitCode=0 Jul 15 23:47:59.665010 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.664981 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerDied","Data":"c1b4ed2c83cf538a86cd0a4173bbb0dfeb6fbe52bef94f837b25edbba62d336c"} Jul 15 23:47:59.921071 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.920987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:47:59.974701 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:47:59.974671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:00.503393 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.503369 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:00.503812 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.503369 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:00.503812 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:00.503460 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:48:00.503812 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:00.503546 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:48:00.670242 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.670205 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" event={"ID":"5b15672f-ce5c-470f-bfd0-c756d4d3f910","Type":"ContainerStarted","Data":"04ebc0a91058e829d2397ea4adefccd1c87c3fc111c980e0c455204ad11c50b5"} Jul 15 23:48:00.692150 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.692113 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vmwm4" podStartSLOduration=15.549173955 podStartE2EDuration="34.692098257s" totalImagesPullingTime="19.142924302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:00.69060657 +0000 UTC m=+35.745936643" watchObservedRunningTime="2026-07-15 23:48:00.692098257 +0000 UTC m=+35.747428331" Jul 15 23:48:00.811016 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.810940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:00.911615 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:00.911592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:01.502925 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:01.502900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:48:01.503091 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:01.503003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wrv5h" podUID="7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9" Jul 15 23:48:02.344570 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.344538 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:02.445476 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.445441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:02.503618 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.503572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:02.503767 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.503574 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:02.503767 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:02.503674 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxp97" podUID="6ab8a1c8-57e8-47b4-999b-63f569069d7a" Jul 15 23:48:02.503850 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:02.503766 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-266qn" podUID="5e664898-1dcd-4d36-9a2e-9fcc70c79725" Jul 15 23:48:02.738800 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.738781 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-211.ec2.internal" event="NodeReady" Jul 15 23:48:02.738900 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.738874 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 23:48:02.781627 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.781609 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c4pkn"] Jul 15 23:48:02.802212 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.802192 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fcvxr"] Jul 15 23:48:02.802354 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.802336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:02.804745 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.804726 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 23:48:02.804845 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.804789 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72cf6\"" Jul 15 23:48:02.805158 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.805139 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 23:48:02.817439 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.817421 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4pkn"] Jul 15 23:48:02.817520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.817452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcvxr"] Jul 15 23:48:02.817520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.817478 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7ps92"] Jul 15 23:48:02.817594 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.817529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.819939 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.819920 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 23:48:02.820265 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.820248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 23:48:02.820362 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.820252 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q4vkt\"" Jul 15 23:48:02.820362 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.820291 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 23:48:02.820463 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.820393 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 23:48:02.833704 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.833673 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ps92"] Jul 15 23:48:02.833815 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.833796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:02.836165 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.836147 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 23:48:02.836274 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.836248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 23:48:02.836274 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.836269 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jq9d4\"" Jul 15 23:48:02.836394 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.836376 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 23:48:02.907945 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.907896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:02.952928 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.952907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljtfd\" (UniqueName: \"kubernetes.io/projected/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-kube-api-access-ljtfd\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:02.952991 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.952932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eef93151-d711-40ad-bb7a-8c6a4e52f08d-data-volume\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.952991 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.952948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdpvr\" (UniqueName: \"kubernetes.io/projected/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-api-access-cdpvr\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.952991 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.952964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.952991 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.952981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eef93151-d711-40ad-bb7a-8c6a4e52f08d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.953129 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eef93151-d711-40ad-bb7a-8c6a4e52f08d-crio-socket\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:02.953129 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-config-volume\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:02.953129 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eca580ce-12b8-42b5-b4b6-1cba272266ed-cert\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:02.953215 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76nf\" (UniqueName: \"kubernetes.io/projected/eca580ce-12b8-42b5-b4b6-1cba272266ed-kube-api-access-g76nf\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:02.953215 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-tmp-dir\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:02.953289 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.953256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-metrics-tls\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:02.961071 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:02.961044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:03.053761 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eef93151-d711-40ad-bb7a-8c6a4e52f08d-crio-socket\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.053855 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-config-volume\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.053855 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eca580ce-12b8-42b5-b4b6-1cba272266ed-cert\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:03.053855 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76nf\" (UniqueName: \"kubernetes.io/projected/eca580ce-12b8-42b5-b4b6-1cba272266ed-kube-api-access-g76nf\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-tmp-dir\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-metrics-tls\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljtfd\" (UniqueName: \"kubernetes.io/projected/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-kube-api-access-ljtfd\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eef93151-d711-40ad-bb7a-8c6a4e52f08d-crio-socket\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eef93151-d711-40ad-bb7a-8c6a4e52f08d-data-volume\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.054008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.053991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdpvr\" (UniqueName: \"kubernetes.io/projected/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-api-access-cdpvr\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.055180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.054358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.055180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.054566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eef93151-d711-40ad-bb7a-8c6a4e52f08d-data-volume\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.055180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.054588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eef93151-d711-40ad-bb7a-8c6a4e52f08d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.055180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.054595 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-tmp-dir\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.055180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.054880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-config-volume\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.055593 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.055571 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.059405 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.059385 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eca580ce-12b8-42b5-b4b6-1cba272266ed-cert\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:03.059937 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.059914 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-metrics-tls\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.060035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.059919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eef93151-d711-40ad-bb7a-8c6a4e52f08d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.061934 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.061912 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljtfd\" (UniqueName: \"kubernetes.io/projected/088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1-kube-api-access-ljtfd\") pod \"dns-default-c4pkn\" (UID: \"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1\") " pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.062138 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.062117 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdpvr\" (UniqueName: \"kubernetes.io/projected/eef93151-d711-40ad-bb7a-8c6a4e52f08d-kube-api-access-cdpvr\") pod \"insights-runtime-extractor-fcvxr\" (UID: \"eef93151-d711-40ad-bb7a-8c6a4e52f08d\") " pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.062592 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.062576 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76nf\" (UniqueName: \"kubernetes.io/projected/eca580ce-12b8-42b5-b4b6-1cba272266ed-kube-api-access-g76nf\") pod \"ingress-canary-7ps92\" (UID: \"eca580ce-12b8-42b5-b4b6-1cba272266ed\") " pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:03.112566 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.112545 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:03.125663 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.125646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcvxr" Jul 15 23:48:03.142080 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.142057 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ps92" Jul 15 23:48:03.312564 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.312532 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4pkn"] Jul 15 23:48:03.316796 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:03.316757 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088d1414_8bf4_4f20_bb7f_f1b2dc7e72b1.slice/crio-7cecb6debe41dbf6690b091bf23d7365054a080e6a5c2ec8d65d9e23cc84abd7 WatchSource:0}: Error finding container 7cecb6debe41dbf6690b091bf23d7365054a080e6a5c2ec8d65d9e23cc84abd7: Status 404 returned error can't find the container with id 7cecb6debe41dbf6690b091bf23d7365054a080e6a5c2ec8d65d9e23cc84abd7 Jul 15 23:48:03.318381 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.318354 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ps92"] Jul 15 23:48:03.320943 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:03.320915 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca580ce_12b8_42b5_b4b6_1cba272266ed.slice/crio-e93b53bb40b23a2c84a8d21e004ab02b5bc49cf470e67aa31a0f2f7ad0329300 WatchSource:0}: Error finding container e93b53bb40b23a2c84a8d21e004ab02b5bc49cf470e67aa31a0f2f7ad0329300: Status 404 returned error can't find the container with id e93b53bb40b23a2c84a8d21e004ab02b5bc49cf470e67aa31a0f2f7ad0329300 Jul 15 23:48:03.322545 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.322521 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcvxr"] Jul 15 23:48:03.325315 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:03.325291 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef93151_d711_40ad_bb7a_8c6a4e52f08d.slice/crio-61e37677c7cb07ca21b8f904d75187e7a2d04085976dc2b2428713744ec3f94c WatchSource:0}: Error finding container 61e37677c7cb07ca21b8f904d75187e7a2d04085976dc2b2428713744ec3f94c: Status 404 returned error can't find the container with id 61e37677c7cb07ca21b8f904d75187e7a2d04085976dc2b2428713744ec3f94c Jul 15 23:48:03.502758 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.502738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:48:03.505454 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.505432 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 23:48:03.505536 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.505521 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m2fgm\"" Jul 15 23:48:03.679538 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.679508 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ps92" event={"ID":"eca580ce-12b8-42b5-b4b6-1cba272266ed","Type":"ContainerStarted","Data":"e93b53bb40b23a2c84a8d21e004ab02b5bc49cf470e67aa31a0f2f7ad0329300"} Jul 15 23:48:03.680523 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.680500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcvxr" event={"ID":"eef93151-d711-40ad-bb7a-8c6a4e52f08d","Type":"ContainerStarted","Data":"61e37677c7cb07ca21b8f904d75187e7a2d04085976dc2b2428713744ec3f94c"} Jul 15 23:48:03.681451 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:03.681432 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4pkn" event={"ID":"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1","Type":"ContainerStarted","Data":"7cecb6debe41dbf6690b091bf23d7365054a080e6a5c2ec8d65d9e23cc84abd7"} Jul 15 23:48:04.252593 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.252560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:04.305183 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.305140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:04.503679 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.503598 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:04.504161 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.503821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:04.506459 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.506437 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 23:48:04.507527 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.507508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fwsxh\"" Jul 15 23:48:04.507527 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.507524 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 23:48:04.507720 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:04.507571 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 23:48:05.951942 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.951692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:05.965626 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.965600 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:05.986107 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.986081 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:05.986263 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.986181 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:05.989308 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.989136 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 23:48:05.989308 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.989172 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 23:48:05.989308 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.989194 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 23:48:05.989898 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.989742 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 23:48:05.990769 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.990517 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-66hns\"" Jul 15 23:48:05.990769 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.990622 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 23:48:05.990769 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.990523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 23:48:05.990973 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:05.990845 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 23:48:06.022241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.022194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:06.075014 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.074991 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:06.075979 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.075945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.076092 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.075994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.076092 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.076020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvnwq\" (UniqueName: \"kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.076184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.076086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.076258 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.076199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.076321 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.076259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177514 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177622 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvnwq\" (UniqueName: \"kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177622 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177740 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177801 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.177801 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.177795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.178168 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.178120 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.178488 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.178466 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.178699 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.178678 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.181981 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.181949 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.181981 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.181962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.185676 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.185634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvnwq\" (UniqueName: \"kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq\") pod \"console-6cfd9c65c8-tzsp7\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:06.302635 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:06.302607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:07.122928 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.122896 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:48:07.129391 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.129358 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:07.180431 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.180410 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:48:07.180552 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.180500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.185572 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.185548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:07.189574 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.189551 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 23:48:07.285521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285681 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285681 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285681 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c69rn\" (UniqueName: \"kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285681 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285674 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285900 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.285900 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.285792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387131 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387131 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387451 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c69rn\" (UniqueName: \"kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387451 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387451 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.387913 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.388049 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.387982 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.388115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.388067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.388115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.388106 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.389865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.389844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.390126 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.390100 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.395602 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.395579 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c69rn\" (UniqueName: \"kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn\") pod \"console-6876bfb45b-bn926\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:07.491789 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:07.491758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:09.527195 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:09.527160 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:09.578610 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:09.578577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:09.719128 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:09.719089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:09.982249 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:09.982199 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1e4809_bc1b_4804_8b91_2197c98f04f4.slice/crio-1e52400905021085abd06657819fd8e6c7930f8bd0ac007312b315f1e2ed0def WatchSource:0}: Error finding container 1e52400905021085abd06657819fd8e6c7930f8bd0ac007312b315f1e2ed0def: Status 404 returned error can't find the container with id 1e52400905021085abd06657819fd8e6c7930f8bd0ac007312b315f1e2ed0def Jul 15 23:48:10.107179 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.106974 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:48:10.110107 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:10.110081 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901b1679_4f2f_46bf_9b83_af87e88cbc8f.slice/crio-457e218bb1c0cf20f371859d4581f412c16f5ccee0d335ee5437dff0b9fd2207 WatchSource:0}: Error finding container 457e218bb1c0cf20f371859d4581f412c16f5ccee0d335ee5437dff0b9fd2207: Status 404 returned error can't find the container with id 457e218bb1c0cf20f371859d4581f412c16f5ccee0d335ee5437dff0b9fd2207 Jul 15 23:48:10.541612 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.541541 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:10.596181 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.596138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:10.701900 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.701855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ps92" event={"ID":"eca580ce-12b8-42b5-b4b6-1cba272266ed","Type":"ContainerStarted","Data":"2efc3858d4ae9a47217f63cc708a015891577a6955f2e69935ba4c882fc0676d"} Jul 15 23:48:10.703542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.703513 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfd9c65c8-tzsp7" event={"ID":"1c1e4809-bc1b-4804-8b91-2197c98f04f4","Type":"ContainerStarted","Data":"1e52400905021085abd06657819fd8e6c7930f8bd0ac007312b315f1e2ed0def"} Jul 15 23:48:10.705655 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.705633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4pkn" event={"ID":"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1","Type":"ContainerStarted","Data":"956189c794c007a881645c27f44013435c6427ea020b8c6218d4746705a7c573"} Jul 15 23:48:10.705751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.705679 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4pkn" event={"ID":"088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1","Type":"ContainerStarted","Data":"450e968b850c468c275e39fd5202ab8e08a9e54e6d3cd20c6e0c366235163cfb"} Jul 15 23:48:10.705751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.705727 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:10.708327 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.708305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcvxr" event={"ID":"eef93151-d711-40ad-bb7a-8c6a4e52f08d","Type":"ContainerStarted","Data":"72a7306cdccc9ec7b57d33c3da6a7599719f4f6dacdf54ef1b470c1a23c3cf9a"} Jul 15 23:48:10.709906 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.709876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6876bfb45b-bn926" event={"ID":"901b1679-4f2f-46bf-9b83-af87e88cbc8f","Type":"ContainerStarted","Data":"457e218bb1c0cf20f371859d4581f412c16f5ccee0d335ee5437dff0b9fd2207"} Jul 15 23:48:10.717396 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.717347 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7ps92" podStartSLOduration=2.060267833 podStartE2EDuration="8.717330276s" totalImagesPullingTime="6.657062443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:10.716587993 +0000 UTC m=+45.771918090" watchObservedRunningTime="2026-07-15 23:48:10.717330276 +0000 UTC m=+45.772660334" Jul 15 23:48:10.737501 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:10.737451 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c4pkn" podStartSLOduration=2.075894612 podStartE2EDuration="8.737433974s" totalImagesPullingTime="6.661539362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:10.736440639 +0000 UTC m=+45.791770714" watchObservedRunningTime="2026-07-15 23:48:10.737433974 +0000 UTC m=+45.792764049" Jul 15 23:48:11.742857 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:11.741209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:11.799714 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:11.799621 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:11.954705 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:11.954670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.056105 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.055968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.437264 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.437167 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.538843 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.538796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.616368 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.616340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.847375 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.847334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:12.978927 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:12.978891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:13.038990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:13.038961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:15.726680 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:15.726580 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcvxr" event={"ID":"eef93151-d711-40ad-bb7a-8c6a4e52f08d","Type":"ContainerStarted","Data":"6b5f88c09419dde17f123104a7fb2586c3c9fc8ec8a77eb2cb2307b0eae5757c"} Jul 15 23:48:15.728371 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:15.728335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6876bfb45b-bn926" event={"ID":"901b1679-4f2f-46bf-9b83-af87e88cbc8f","Type":"ContainerStarted","Data":"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07"} Jul 15 23:48:15.729994 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:15.729971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfd9c65c8-tzsp7" event={"ID":"1c1e4809-bc1b-4804-8b91-2197c98f04f4","Type":"ContainerStarted","Data":"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf"} Jul 15 23:48:15.748300 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:15.748251 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6876bfb45b-bn926" podStartSLOduration=3.597237393 podStartE2EDuration="8.74823353s" totalImagesPullingTime="5.150996137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:15.747490955 +0000 UTC m=+50.802821029" watchObservedRunningTime="2026-07-15 23:48:15.74823353 +0000 UTC m=+50.803563599" Jul 15 23:48:15.771944 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:15.771905 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cfd9c65c8-tzsp7" podStartSLOduration=5.492843036 podStartE2EDuration="10.771887691s" totalImagesPullingTime="5.279044655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:15.771075806 +0000 UTC m=+50.826405880" watchObservedRunningTime="2026-07-15 23:48:15.771887691 +0000 UTC m=+50.827217766" Jul 15 23:48:16.302782 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.302746 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:16.302943 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.302802 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:16.304203 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.304173 2580 patch_prober.go:28] interesting pod/console-6cfd9c65c8-tzsp7 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 15 23:48:16.304358 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.304239 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6cfd9c65c8-tzsp7" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 15 23:48:16.438538 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.438505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:16.629696 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.629616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:16.692896 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.692868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:16.823993 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:16.823960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:17.491985 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:17.491929 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:17.492180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:17.492049 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:17.494013 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:17.493654 2580 patch_prober.go:28] interesting pod/console-6876bfb45b-bn926 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body= Jul 15 23:48:17.494013 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:17.493719 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6876bfb45b-bn926" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" Jul 15 23:48:18.742385 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:18.742360 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcvxr" event={"ID":"eef93151-d711-40ad-bb7a-8c6a4e52f08d","Type":"ContainerStarted","Data":"1cec4553e53b5a6a7499ae9a0424d9c4c5f5998ce42ad5d3b489f3c1b554735d"} Jul 15 23:48:18.760568 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:18.760528 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fcvxr" podStartSLOduration=1.586172902 podStartE2EDuration="16.760512902s" totalImagesPullingTime="15.17434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:48:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:18.759896226 +0000 UTC m=+53.815226309" watchObservedRunningTime="2026-07-15 23:48:18.760512902 +0000 UTC m=+53.815843004" Jul 15 23:48:18.955691 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:18.955636 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:19.014687 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:19.014667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:19.249617 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:19.249596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:19.310374 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:19.310354 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:19.585398 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:19.585340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:20.716062 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:20.716036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c4pkn" Jul 15 23:48:20.761735 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:20.761709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:20.983842 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:20.983778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.010709 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.010682 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz"] Jul 15 23:48:21.045985 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.045966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-56kxs"] Jul 15 23:48:21.046130 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.046110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.048844 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.048824 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 23:48:21.049080 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.049068 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 23:48:21.049180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.049160 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 23:48:21.049308 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.049240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 23:48:21.049308 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.049287 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 23:48:21.049536 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.049378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6mld5\"" Jul 15 23:48:21.057334 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.057313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.060210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.060197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz"] Jul 15 23:48:21.060269 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.060257 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gdjm5"] Jul 15 23:48:21.060397 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.060383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.065845 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.065771 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 23:48:21.066562 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.066538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9zl2r\"" Jul 15 23:48:21.066656 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.066625 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 23:48:21.067200 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.067185 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 23:48:21.077405 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.077388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-56kxs"] Jul 15 23:48:21.077508 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.077496 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.080016 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.079999 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 23:48:21.080108 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.080030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 23:48:21.080108 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.080005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fvd28\"" Jul 15 23:48:21.080188 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.080132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 23:48:21.125355 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.125331 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.195746 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r2pl\" (UniqueName: \"kubernetes.io/projected/1ee6eb64-d6c4-45a6-a835-35829f78882b-kube-api-access-5r2pl\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.195838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.195838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-textfile\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.195838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-metrics-client-ca\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.195838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz4qg\" (UniqueName: \"kubernetes.io/projected/d63be52a-76c0-4254-a95a-1cd47fe6692c-kube-api-access-gz4qg\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.195990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-wtmp\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.195990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-root\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.195990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98b956d6-7d07-4587-a659-6fba4587f78f-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.195990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-sys\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.195997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-tls\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196026 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.196115 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee6eb64-d6c4-45a6-a835-35829f78882b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.196334 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drzx2\" (UniqueName: \"kubernetes.io/projected/98b956d6-7d07-4587-a659-6fba4587f78f-kube-api-access-drzx2\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.196334 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.196334 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.196334 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.196249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.227927 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.227901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.297201 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297177 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-root\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297348 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98b956d6-7d07-4587-a659-6fba4587f78f-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297348 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-sys\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297348 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-tls\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297348 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-root\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297348 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297373 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-sys\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee6eb64-d6c4-45a6-a835-35829f78882b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drzx2\" (UniqueName: \"kubernetes.io/projected/98b956d6-7d07-4587-a659-6fba4587f78f-kube-api-access-drzx2\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.297583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98b956d6-7d07-4587-a659-6fba4587f78f-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r2pl\" (UniqueName: \"kubernetes.io/projected/1ee6eb64-d6c4-45a6-a835-35829f78882b-kube-api-access-5r2pl\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-textfile\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-metrics-client-ca\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz4qg\" (UniqueName: \"kubernetes.io/projected/d63be52a-76c0-4254-a95a-1cd47fe6692c-kube-api-access-gz4qg\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-wtmp\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.297956 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.297926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-wtmp\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.298343 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.298113 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.298343 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.298201 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee6eb64-d6c4-45a6-a835-35829f78882b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.298570 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.298521 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.298919 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.298636 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-metrics-client-ca\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.299053 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.298963 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-textfile\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.299267 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.299235 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.301407 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.301407 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301375 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.301555 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.301622 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98b956d6-7d07-4587-a659-6fba4587f78f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.301676 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ee6eb64-d6c4-45a6-a835-35829f78882b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.301727 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.301693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d63be52a-76c0-4254-a95a-1cd47fe6692c-node-exporter-tls\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.304887 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.304868 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r2pl\" (UniqueName: \"kubernetes.io/projected/1ee6eb64-d6c4-45a6-a835-35829f78882b-kube-api-access-5r2pl\") pod \"openshift-state-metrics-84bd68b896-bxxhz\" (UID: \"1ee6eb64-d6c4-45a6-a835-35829f78882b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.305190 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.305170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drzx2\" (UniqueName: \"kubernetes.io/projected/98b956d6-7d07-4587-a659-6fba4587f78f-kube-api-access-drzx2\") pod \"kube-state-metrics-785778ff74-56kxs\" (UID: \"98b956d6-7d07-4587-a659-6fba4587f78f\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.305646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.305607 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz4qg\" (UniqueName: \"kubernetes.io/projected/d63be52a-76c0-4254-a95a-1cd47fe6692c-kube-api-access-gz4qg\") pod \"node-exporter-gdjm5\" (UID: \"d63be52a-76c0-4254-a95a-1cd47fe6692c\") " pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.327441 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.327399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.355300 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.355277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" Jul 15 23:48:21.371430 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.371412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" Jul 15 23:48:21.380573 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.380545 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.385521 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.385503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdjm5" Jul 15 23:48:21.395838 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:21.395810 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63be52a_76c0_4254_a95a_1cd47fe6692c.slice/crio-2a88557e39df1b97ff1a1fd92fd130504a4ee41817d17858a6e2d175b8433c8c WatchSource:0}: Error finding container 2a88557e39df1b97ff1a1fd92fd130504a4ee41817d17858a6e2d175b8433c8c: Status 404 returned error can't find the container with id 2a88557e39df1b97ff1a1fd92fd130504a4ee41817d17858a6e2d175b8433c8c Jul 15 23:48:21.490794 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.490766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz"] Jul 15 23:48:21.494577 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:21.494552 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee6eb64_d6c4_45a6_a835_35829f78882b.slice/crio-440fc8b11ca4bf46737a8f2fe76ef0d7f9599870f5986c2d4a9f470f939bc43a WatchSource:0}: Error finding container 440fc8b11ca4bf46737a8f2fe76ef0d7f9599870f5986c2d4a9f470f939bc43a: Status 404 returned error can't find the container with id 440fc8b11ca4bf46737a8f2fe76ef0d7f9599870f5986c2d4a9f470f939bc43a Jul 15 23:48:21.507330 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.507308 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-56kxs"] Jul 15 23:48:21.509565 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:21.509543 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b956d6_7d07_4587_a659_6fba4587f78f.slice/crio-22d79ae4821e4226d0951ef6bfd426a70211f866a7e8863af55129d4570e8b46 WatchSource:0}: Error finding container 22d79ae4821e4226d0951ef6bfd426a70211f866a7e8863af55129d4570e8b46: Status 404 returned error can't find the container with id 22d79ae4821e4226d0951ef6bfd426a70211f866a7e8863af55129d4570e8b46 Jul 15 23:48:21.554550 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.554519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.654481 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.654453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:21.754836 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.754810 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdjm5" event={"ID":"d63be52a-76c0-4254-a95a-1cd47fe6692c","Type":"ContainerStarted","Data":"2a88557e39df1b97ff1a1fd92fd130504a4ee41817d17858a6e2d175b8433c8c"} Jul 15 23:48:21.756572 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.756543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" event={"ID":"1ee6eb64-d6c4-45a6-a835-35829f78882b","Type":"ContainerStarted","Data":"9818012d8c4d8b4fa45d088a1d05b86f769e8ff1c515bb0eda1431ff19f776c1"} Jul 15 23:48:21.756672 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.756591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" event={"ID":"1ee6eb64-d6c4-45a6-a835-35829f78882b","Type":"ContainerStarted","Data":"e741b211e3cbc46dc150e0e17be36293c1162e33b0c8dc58129a3d58a90c9306"} Jul 15 23:48:21.756672 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.756616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" event={"ID":"1ee6eb64-d6c4-45a6-a835-35829f78882b","Type":"ContainerStarted","Data":"440fc8b11ca4bf46737a8f2fe76ef0d7f9599870f5986c2d4a9f470f939bc43a"} Jul 15 23:48:21.757569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.757549 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" event={"ID":"98b956d6-7d07-4587-a659-6fba4587f78f","Type":"ContainerStarted","Data":"22d79ae4821e4226d0951ef6bfd426a70211f866a7e8863af55129d4570e8b46"} Jul 15 23:48:21.966991 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:21.966921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:22.072513 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.072481 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:22.076966 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.076937 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 23:48:22.079421 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.079403 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.082029 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082007 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 23:48:22.082144 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 23:48:22.082144 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082140 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 23:48:22.082276 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082133 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 23:48:22.082276 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082142 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 23:48:22.082371 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082315 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 23:48:22.082558 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 23:48:22.082645 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082556 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 23:48:22.082645 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082574 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 23:48:22.082720 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.082658 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6xr6n\"" Jul 15 23:48:22.097573 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.097552 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 23:48:22.161760 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.161722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-web-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206082 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm6pq\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-kube-api-access-dm6pq\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.205998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.206204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-config-out\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.206258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.206325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.206366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.206537 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.206423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.307577 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.307745 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.307745 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.307745 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-web-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.307745 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm6pq\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-kube-api-access-dm6pq\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-config-out\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308003 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.307985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308395 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:22.308056 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle podName:1c657173-40bb-488d-acce-8a6097d30328 nodeName:}" failed. No retries permitted until 2026-07-15 23:48:22.808031049 +0000 UTC m=+57.863361121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1c657173-40bb-488d-acce-8a6097d30328") : configmap references non-existent config key: ca-bundle.crt Jul 15 23:48:22.308395 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.308089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.308848 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.308822 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.313802 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.313848 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.314381 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.315792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c657173-40bb-488d-acce-8a6097d30328-config-out\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.315898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.316076 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.316168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-web-config\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.316971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.316420 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.318087 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.317986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c657173-40bb-488d-acce-8a6097d30328-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.324917 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.324899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm6pq\" (UniqueName: \"kubernetes.io/projected/1c657173-40bb-488d-acce-8a6097d30328-kube-api-access-dm6pq\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.763915 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.763879 2580 generic.go:356] "Generic (PLEG): container finished" podID="d63be52a-76c0-4254-a95a-1cd47fe6692c" containerID="dd487b2678369f04677e533c5e326ec9159acbd36ac73cde30b8e16e1e6eb46f" exitCode=0 Jul 15 23:48:22.764339 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.763953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdjm5" event={"ID":"d63be52a-76c0-4254-a95a-1cd47fe6692c","Type":"ContainerDied","Data":"dd487b2678369f04677e533c5e326ec9159acbd36ac73cde30b8e16e1e6eb46f"} Jul 15 23:48:22.814144 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.814117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.814950 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.814931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c657173-40bb-488d-acce-8a6097d30328-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c657173-40bb-488d-acce-8a6097d30328\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:22.990498 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:22.990475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 23:48:23.169660 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.169556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 23:48:23.171724 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:23.171696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c657173_40bb_488d_acce_8a6097d30328.slice/crio-2a3b327e95ae252cbba075789ddb5aeb6468bfbe1dc298fb0e49897e06ec274b WatchSource:0}: Error finding container 2a3b327e95ae252cbba075789ddb5aeb6468bfbe1dc298fb0e49897e06ec274b: Status 404 returned error can't find the container with id 2a3b327e95ae252cbba075789ddb5aeb6468bfbe1dc298fb0e49897e06ec274b Jul 15 23:48:23.774668 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.774631 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" event={"ID":"98b956d6-7d07-4587-a659-6fba4587f78f","Type":"ContainerStarted","Data":"79ea062abddc64eb5b99fe3471c87b79f449c6cd32d8db181a539f5f9d41237e"} Jul 15 23:48:23.775099 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.774726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" event={"ID":"98b956d6-7d07-4587-a659-6fba4587f78f","Type":"ContainerStarted","Data":"78ba299a1626eafb6405f3dc1eaac1ef7cc381a1c5422811b73a462ea9acd17d"} Jul 15 23:48:23.775099 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.774757 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" event={"ID":"98b956d6-7d07-4587-a659-6fba4587f78f","Type":"ContainerStarted","Data":"4946ee4d3dcceabcea7e23f91991dd24df4dcf3a1346d0a438fd1e14e72cfdc1"} Jul 15 23:48:23.776096 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.776066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"2a3b327e95ae252cbba075789ddb5aeb6468bfbe1dc298fb0e49897e06ec274b"} Jul 15 23:48:23.778538 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.778511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdjm5" event={"ID":"d63be52a-76c0-4254-a95a-1cd47fe6692c","Type":"ContainerStarted","Data":"2c15a7fe3c9094b7c39826b8a618bf5e50085594d240ab27db7e5d78d0cac92e"} Jul 15 23:48:23.778649 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.778563 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdjm5" event={"ID":"d63be52a-76c0-4254-a95a-1cd47fe6692c","Type":"ContainerStarted","Data":"6eb9a837924a809bd61473c723ed0c177a1b3d8c9f3c31a59b344615ba47b876"} Jul 15 23:48:23.780846 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.780823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" event={"ID":"1ee6eb64-d6c4-45a6-a835-35829f78882b","Type":"ContainerStarted","Data":"2d7acbc74c194c874b589edc3ddf21ac215a059365ac96966bd94c7e0008855c"} Jul 15 23:48:23.792277 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.792213 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-56kxs" podStartSLOduration=1.322186267 podStartE2EDuration="2.792201054s" totalImagesPullingTime="1.470014787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:23.791037558 +0000 UTC m=+58.846367681" watchObservedRunningTime="2026-07-15 23:48:23.792201054 +0000 UTC m=+58.847531105" Jul 15 23:48:23.809381 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.809338 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bxxhz" podStartSLOduration=2.431470225 podStartE2EDuration="3.809323461s" totalImagesPullingTime="1.377853236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:23.808435642 +0000 UTC m=+58.863765715" watchObservedRunningTime="2026-07-15 23:48:23.809323461 +0000 UTC m=+58.864653533" Jul 15 23:48:23.831444 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.831400 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gdjm5" podStartSLOduration=1.812269031 podStartE2EDuration="2.831383887s" totalImagesPullingTime="942.792753ms" totalInitContainerRuntime="76.322103ms" isStatefulPod=true podCreationTimestamp="2026-07-15 23:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:23.830964985 +0000 UTC m=+58.886295062" watchObservedRunningTime="2026-07-15 23:48:23.831383887 +0000 UTC m=+58.886713961" Jul 15 23:48:23.947705 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:23.947677 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.017041 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.017003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.177540 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.177460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.390421 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.390376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.570161 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.570129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.622872 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.622837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.703773 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.703740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.756615 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.756593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:24.785316 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.785301 2580 generic.go:356] "Generic (PLEG): container finished" podID="1c657173-40bb-488d-acce-8a6097d30328" containerID="563a33a52b423db280e35f5644d809992e1e98f96e1d948bb051904246094c30" exitCode=0 Jul 15 23:48:24.785614 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:24.785408 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerDied","Data":"563a33a52b423db280e35f5644d809992e1e98f96e1d948bb051904246094c30"} Jul 15 23:48:25.307320 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.307289 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5fdcb84974-nknxh"] Jul 15 23:48:25.309772 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.309754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.312369 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.312332 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 23:48:25.313548 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.313526 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 23:48:25.313670 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.313565 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 23:48:25.313670 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.313576 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wdv2x\"" Jul 15 23:48:25.313670 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.313526 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c674udtvu33pb\"" Jul 15 23:48:25.313848 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.313832 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 23:48:25.318798 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.318776 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fdcb84974-nknxh"] Jul 15 23:48:25.337648 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337767 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-client-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337767 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337727 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dcf0282f-e149-4a75-afb9-2a5ac96371aa-audit-log\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337767 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337933 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88x2\" (UniqueName: \"kubernetes.io/projected/dcf0282f-e149-4a75-afb9-2a5ac96371aa-kube-api-access-l88x2\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337933 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-metrics-server-audit-profiles\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.337933 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.337886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-tls\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.438981 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.438952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dcf0282f-e149-4a75-afb9-2a5ac96371aa-audit-log\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439112 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.438992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439112 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88x2\" (UniqueName: \"kubernetes.io/projected/dcf0282f-e149-4a75-afb9-2a5ac96371aa-kube-api-access-l88x2\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439112 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-metrics-server-audit-profiles\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439327 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-tls\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439327 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439327 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-client-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439491 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dcf0282f-e149-4a75-afb9-2a5ac96371aa-audit-log\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.439802 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.439759 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.440327 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.440295 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dcf0282f-e149-4a75-afb9-2a5ac96371aa-metrics-server-audit-profiles\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.441949 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.441928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.442392 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.442371 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-secret-metrics-server-tls\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.442477 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.442428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dcf0282f-e149-4a75-afb9-2a5ac96371aa-client-ca-bundle\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.453721 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.453698 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88x2\" (UniqueName: \"kubernetes.io/projected/dcf0282f-e149-4a75-afb9-2a5ac96371aa-kube-api-access-l88x2\") pod \"metrics-server-5fdcb84974-nknxh\" (UID: \"dcf0282f-e149-4a75-afb9-2a5ac96371aa\") " pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.621409 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.621342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:25.660594 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.660554 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q456c" Jul 15 23:48:25.766463 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.766435 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fdcb84974-nknxh"] Jul 15 23:48:25.813593 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.813565 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:25.858504 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.858472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:25.908280 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:25.908158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:26.065900 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:26.065870 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf0282f_e149_4a75_afb9_2a5ac96371aa.slice/crio-d10fc7abb7b52334f55736ed116b89a8a2585d3032fbd2ca75327ed257a2c16f WatchSource:0}: Error finding container d10fc7abb7b52334f55736ed116b89a8a2585d3032fbd2ca75327ed257a2c16f: Status 404 returned error can't find the container with id d10fc7abb7b52334f55736ed116b89a8a2585d3032fbd2ca75327ed257a2c16f Jul 15 23:48:26.201746 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.201720 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5887f979f7-7jc9c"] Jul 15 23:48:26.205555 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.205534 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.208641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208624 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vs4wp\"" Jul 15 23:48:26.208741 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208688 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 23:48:26.208741 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208693 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 23:48:26.208899 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208882 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 23:48:26.208969 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208882 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 23:48:26.208969 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.208958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 23:48:26.213730 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.213713 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 23:48:26.217583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.217564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5887f979f7-7jc9c"] Jul 15 23:48:26.245418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-serving-certs-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-telemeter-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl84z\" (UniqueName: \"kubernetes.io/projected/73859a5b-1b20-435d-9683-49b92814504d-kube-api-access-pl84z\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245629 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245693 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245731 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245771 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-federate-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.245811 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.245780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-metrics-client-ca\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346593 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-federate-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346871 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-metrics-client-ca\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346871 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346814 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-serving-certs-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.346871 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-telemeter-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.347009 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.346878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl84z\" (UniqueName: \"kubernetes.io/projected/73859a5b-1b20-435d-9683-49b92814504d-kube-api-access-pl84z\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.347927 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.347897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-serving-certs-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.348039 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.347933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-metrics-client-ca\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.348680 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.348655 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73859a5b-1b20-435d-9683-49b92814504d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.349854 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.349799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.350194 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.350171 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-federate-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.350194 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.350189 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-telemeter-client-tls\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.350653 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.350633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73859a5b-1b20-435d-9683-49b92814504d-secret-telemeter-client\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.354389 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.354369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl84z\" (UniqueName: \"kubernetes.io/projected/73859a5b-1b20-435d-9683-49b92814504d-kube-api-access-pl84z\") pod \"telemeter-client-5887f979f7-7jc9c\" (UID: \"73859a5b-1b20-435d-9683-49b92814504d\") " pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.401653 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.401627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:26.529997 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.529977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" Jul 15 23:48:26.657613 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.657586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:26.657764 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.657744 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5887f979f7-7jc9c"] Jul 15 23:48:26.660924 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:26.660902 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73859a5b_1b20_435d_9683_49b92814504d.slice/crio-a4456d65b4c3d5865087fe046a6726b08bd96a6d0f946f064bc3251d9ef61058 WatchSource:0}: Error finding container a4456d65b4c3d5865087fe046a6726b08bd96a6d0f946f064bc3251d9ef61058: Status 404 returned error can't find the container with id a4456d65b4c3d5865087fe046a6726b08bd96a6d0f946f064bc3251d9ef61058 Jul 15 23:48:26.759549 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.759523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:26.797104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.797039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"a57f6c380aef47329799e4fcd28283d0d9fa99dd7100a3acf07b3cb2d3085641"} Jul 15 23:48:26.797214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.797113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"2b665d4bfd672184fdfa04bf12f87b37b1d96e0644a3194a0fc2565581f45cf4"} Jul 15 23:48:26.797214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.797152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"353c2727e2308212829e72aa5f08a35b71c76c867f99e30edee5ae2678556f0e"} Jul 15 23:48:26.797214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.797191 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"10a26fe020876fbaf989edaf57a48108bc140ddaaa2f3460c4338314073408ce"} Jul 15 23:48:26.797394 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.797249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"ebe79f1edc391a309097df61784ce75213dd63a923e70f2a311628c739b206d8"} Jul 15 23:48:26.798403 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.798379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" event={"ID":"73859a5b-1b20-435d-9683-49b92814504d","Type":"ContainerStarted","Data":"a4456d65b4c3d5865087fe046a6726b08bd96a6d0f946f064bc3251d9ef61058"} Jul 15 23:48:26.799789 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:26.799765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" event={"ID":"dcf0282f-e149-4a75-afb9-2a5ac96371aa","Type":"ContainerStarted","Data":"d10fc7abb7b52334f55736ed116b89a8a2585d3032fbd2ca75327ed257a2c16f"} Jul 15 23:48:27.215696 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.215575 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 23:48:27.218008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.217925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:27.227450 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.227429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.230727 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.230513 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 23:48:27.230727 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.230719 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 23:48:27.231052 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231030 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 23:48:27.231052 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231046 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ztgff\"" Jul 15 23:48:27.231817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231391 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e4hs3kdofnm2u\"" Jul 15 23:48:27.231817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231417 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 23:48:27.231817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231530 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 23:48:27.231817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231595 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 23:48:27.231817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231684 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 23:48:27.232147 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.231926 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 23:48:27.233546 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.232294 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 23:48:27.233546 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.232634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 23:48:27.233546 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.232648 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 23:48:27.233546 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.232814 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 23:48:27.233546 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.232933 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 23:48:27.239425 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.239317 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 23:48:27.239525 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.239462 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 23:48:27.252866 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.252845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253002 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.252981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253103 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253103 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253242 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253242 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253242 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253242 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253448 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4xd\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-kube-api-access-cn4xd\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253448 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253272 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253448 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253790 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253790 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253683 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253790 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253790 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.253790 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.253774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354280 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4xd\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-kube-api-access-cn4xd\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354474 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.354774 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.354664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.355187 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.355164 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.355496 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.355315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.355799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356146 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356146 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356292 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356292 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356292 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356292 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.356583 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.356557 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.357123 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.357097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.358902 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.357876 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.358902 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.358547 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.358902 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.358605 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.359107 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.359071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.359168 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.359090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.359805 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.359758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.360683 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.360656 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.360833 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.360672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.361147 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.361104 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.361273 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.361186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-config-out\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.361405 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.361368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-web-config\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.361999 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.361974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.362459 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.362434 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.373046 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.373025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4xd\" (UniqueName: \"kubernetes.io/projected/a91adafa-f20a-4bbf-8a7d-c6a110bddfb0-kube-api-access-cn4xd\") pod \"prometheus-k8s-0\" (UID: \"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.497421 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.497397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:27.502585 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.502566 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:48:27.541545 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.541521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:27.643732 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.643659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:27.693478 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:27.693453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:28.503228 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:28.503193 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:28.558747 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:28.558717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:28.993296 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:28.993264 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 23:48:28.994641 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:28.994613 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91adafa_f20a_4bbf_8a7d_c6a110bddfb0.slice/crio-13441b2f30da7bcb39e2c3620d3f115f46a67e95777b73c4a617697a45cd497b WatchSource:0}: Error finding container 13441b2f30da7bcb39e2c3620d3f115f46a67e95777b73c4a617697a45cd497b: Status 404 returned error can't find the container with id 13441b2f30da7bcb39e2c3620d3f115f46a67e95777b73c4a617697a45cd497b Jul 15 23:48:29.585060 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.585030 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:29.815642 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.815614 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c657173-40bb-488d-acce-8a6097d30328","Type":"ContainerStarted","Data":"c8edfdd03b5b18ff602584d5d468c80968736f1b30d1256db15360a41662c9e6"} Jul 15 23:48:29.817952 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.817915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" event={"ID":"73859a5b-1b20-435d-9683-49b92814504d","Type":"ContainerStarted","Data":"a6d893c5cfe97f0b9b5bc191b87c3b3580fed4ae2c5361b75c099afd0610d99b"} Jul 15 23:48:29.818087 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.817973 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" event={"ID":"73859a5b-1b20-435d-9683-49b92814504d","Type":"ContainerStarted","Data":"dcfd7d040f0e5d157e2805f34d8cc4ce819a407518e83d9aecdc89c5c33ca90e"} Jul 15 23:48:29.818087 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.818007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" event={"ID":"73859a5b-1b20-435d-9683-49b92814504d","Type":"ContainerStarted","Data":"c5f1006bd1aed2be7635d7c48ff535907910711b0e4c1e7a2ddf3eb83386413c"} Jul 15 23:48:29.819797 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.819773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" event={"ID":"dcf0282f-e149-4a75-afb9-2a5ac96371aa","Type":"ContainerStarted","Data":"030b7ad3352999445fe4f0f3200c703445e09eacd9d1aa555f8690218855ec4a"} Jul 15 23:48:29.822365 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.821495 2580 generic.go:356] "Generic (PLEG): container finished" podID="a91adafa-f20a-4bbf-8a7d-c6a110bddfb0" containerID="03d158167b0d045a7e0d97b19d1890bc00823a387b75d34a408c352ffef1f442" exitCode=0 Jul 15 23:48:29.822365 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.821599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerDied","Data":"03d158167b0d045a7e0d97b19d1890bc00823a387b75d34a408c352ffef1f442"} Jul 15 23:48:29.822365 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.821680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"13441b2f30da7bcb39e2c3620d3f115f46a67e95777b73c4a617697a45cd497b"} Jul 15 23:48:29.844706 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.844632 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.004518054 podStartE2EDuration="7.844618847s" totalImagesPullingTime="4.824622958s" totalInitContainerRuntime="15.477835ms" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:29.842738675 +0000 UTC m=+64.898068760" watchObservedRunningTime="2026-07-15 23:48:29.844618847 +0000 UTC m=+64.899948897" Jul 15 23:48:29.861370 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.861326 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" podStartSLOduration=2.100099487 podStartE2EDuration="4.861312754s" totalImagesPullingTime="2.761213267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:29.860586256 +0000 UTC m=+64.915916332" watchObservedRunningTime="2026-07-15 23:48:29.861312754 +0000 UTC m=+64.916642825" Jul 15 23:48:29.881394 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:29.881355 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5887f979f7-7jc9c" podStartSLOduration=1.654535323 podStartE2EDuration="3.881342484s" totalImagesPullingTime="2.226807161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:29.879636416 +0000 UTC m=+64.934966516" watchObservedRunningTime="2026-07-15 23:48:29.881342484 +0000 UTC m=+64.936672556" Jul 15 23:48:30.729729 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.729695 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:48:30.732092 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.732064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.744192 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.744169 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:48:30.786191 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62rg\" (UniqueName: \"kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786465 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786465 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786566 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.786566 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.786531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887528 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887679 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887679 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887679 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887847 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.887847 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.887842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.888270 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.888213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.888838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.888631 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.888838 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.888780 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.889042 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.888867 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.889042 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.888904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62rg\" (UniqueName: \"kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.890761 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.890737 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.891059 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.891034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:30.896810 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:30.896787 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62rg\" (UniqueName: \"kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg\") pod \"console-7c465c66-f4884\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:31.044204 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.044175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:31.094636 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.094611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:31.181266 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.181232 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:48:31.184624 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.184594 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:31.188185 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:31.188161 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6381bf_5be8_42d9_b83f_692ce5e8a24b.slice/crio-25997e771a97b1beb0eeb7f6a67f8567cbab37d6ec02677836a642dcb3dd92a0 WatchSource:0}: Error finding container 25997e771a97b1beb0eeb7f6a67f8567cbab37d6ec02677836a642dcb3dd92a0: Status 404 returned error can't find the container with id 25997e771a97b1beb0eeb7f6a67f8567cbab37d6ec02677836a642dcb3dd92a0 Jul 15 23:48:31.192106 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.192080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:48:31.194697 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.194628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 23:48:31.205171 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.205149 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9-metrics-certs\") pod \"network-metrics-daemon-wrv5h\" (UID: \"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9\") " pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:48:31.394453 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.394355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:31.397317 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.397295 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 23:48:31.407656 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.407629 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ab8a1c8-57e8-47b4-999b-63f569069d7a-original-pull-secret\") pod \"global-pull-secret-syncer-xxp97\" (UID: \"6ab8a1c8-57e8-47b4-999b-63f569069d7a\") " pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:31.414805 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.414787 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m2fgm\"" Jul 15 23:48:31.423169 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.423149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wrv5h" Jul 15 23:48:31.495603 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.495502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:31.498261 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.498210 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 23:48:31.508367 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.508302 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 23:48:31.517074 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.517046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxp97" Jul 15 23:48:31.519418 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.519396 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn6ws\" (UniqueName: \"kubernetes.io/projected/5e664898-1dcd-4d36-9a2e-9fcc70c79725-kube-api-access-tn6ws\") pod \"network-check-target-266qn\" (UID: \"5e664898-1dcd-4d36-9a2e-9fcc70c79725\") " pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:31.529371 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.529319 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fwsxh\"" Jul 15 23:48:31.535390 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.535356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:31.569819 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.569789 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wrv5h"] Jul 15 23:48:31.572373 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:31.572342 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c02f0d6_b31f_4dfb_a9a0_b49c0b47bdc9.slice/crio-c3d792bdc0ebfb5d79e8eab39cc593b3c01d83f6af162a2479bf605c3995962b WatchSource:0}: Error finding container c3d792bdc0ebfb5d79e8eab39cc593b3c01d83f6af162a2479bf605c3995962b: Status 404 returned error can't find the container with id c3d792bdc0ebfb5d79e8eab39cc593b3c01d83f6af162a2479bf605c3995962b Jul 15 23:48:31.675089 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.675018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxp97"] Jul 15 23:48:31.678573 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:31.678542 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab8a1c8_57e8_47b4_999b_63f569069d7a.slice/crio-f8ae35954253a849f54a248165632f65a1dd4b4b49b5abac4b45e5e3cf250772 WatchSource:0}: Error finding container f8ae35954253a849f54a248165632f65a1dd4b4b49b5abac4b45e5e3cf250772: Status 404 returned error can't find the container with id f8ae35954253a849f54a248165632f65a1dd4b4b49b5abac4b45e5e3cf250772 Jul 15 23:48:31.698516 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.698479 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-266qn"] Jul 15 23:48:31.702305 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:48:31.702279 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e664898_1dcd_4d36_9a2e_9fcc70c79725.slice/crio-ad983859d150f4d1de5f10f0ab67154b21c985acd25d0a1ca1ebb6b82d7eccd3 WatchSource:0}: Error finding container ad983859d150f4d1de5f10f0ab67154b21c985acd25d0a1ca1ebb6b82d7eccd3: Status 404 returned error can't find the container with id ad983859d150f4d1de5f10f0ab67154b21c985acd25d0a1ca1ebb6b82d7eccd3 Jul 15 23:48:31.757150 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.757122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:31.836122 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.836084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c465c66-f4884" event={"ID":"dd6381bf-5be8-42d9-b83f-692ce5e8a24b","Type":"ContainerStarted","Data":"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77"} Jul 15 23:48:31.836267 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.836137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c465c66-f4884" event={"ID":"dd6381bf-5be8-42d9-b83f-692ce5e8a24b","Type":"ContainerStarted","Data":"25997e771a97b1beb0eeb7f6a67f8567cbab37d6ec02677836a642dcb3dd92a0"} Jul 15 23:48:31.837565 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.837537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-266qn" event={"ID":"5e664898-1dcd-4d36-9a2e-9fcc70c79725","Type":"ContainerStarted","Data":"ad983859d150f4d1de5f10f0ab67154b21c985acd25d0a1ca1ebb6b82d7eccd3"} Jul 15 23:48:31.838829 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.838806 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxp97" event={"ID":"6ab8a1c8-57e8-47b4-999b-63f569069d7a","Type":"ContainerStarted","Data":"f8ae35954253a849f54a248165632f65a1dd4b4b49b5abac4b45e5e3cf250772"} Jul 15 23:48:31.840031 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.840007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wrv5h" event={"ID":"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9","Type":"ContainerStarted","Data":"c3d792bdc0ebfb5d79e8eab39cc593b3c01d83f6af162a2479bf605c3995962b"} Jul 15 23:48:31.858606 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.858557 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c465c66-f4884" podStartSLOduration=1.858539902 podStartE2EDuration="1.858539902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:31.856833194 +0000 UTC m=+66.912163272" watchObservedRunningTime="2026-07-15 23:48:31.858539902 +0000 UTC m=+66.913869977" Jul 15 23:48:31.863885 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:31.863862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:32.394339 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:32.394303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:32.584716 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:32.584683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:33.740912 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.740826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:33.844648 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.844587 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:33.852325 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.852281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wrv5h" event={"ID":"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9","Type":"ContainerStarted","Data":"bc89f5419695dba583f666e0e868a832fc6f8ece1a3256d8ed1b2fd409dc452b"} Jul 15 23:48:33.852475 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.852347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wrv5h" event={"ID":"7c02f0d6-b31f-4dfb-a9a0-b49c0b47bdc9","Type":"ContainerStarted","Data":"cd2faebf9daaa337470089408ce11d2a3916cbcb0cb4ef28ca6df232d163ec1b"} Jul 15 23:48:33.856788 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.856662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"affdb0812c306144ec745a1a7bc650462a0111f13463294718b1f8aa792c7bfa"} Jul 15 23:48:33.856788 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.856727 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"f32e7cce7b28b8f467e125245e877fd50f95cd9e6a27c5a2d57c0650f4d92382"} Jul 15 23:48:33.870053 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.870006 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wrv5h" podStartSLOduration=67.145357212 podStartE2EDuration="1m8.869986255s" totalImagesPullingTime="1.724629043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:33.868284021 +0000 UTC m=+68.923614096" watchObservedRunningTime="2026-07-15 23:48:33.869986255 +0000 UTC m=+68.925316329" Jul 15 23:48:33.941061 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:33.941028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:34.040857 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:34.040773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:34.350822 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:34.350704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:34.450872 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:34.450837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:35.743509 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:35.743468 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:35.811178 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:35.811131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:36.112088 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:36.112011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:36.168497 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:36.168464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:38.593603 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.593576 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:38.880662 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.880602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"459a703b767e26248a5e4e7e3af5e016b8be48fc370f6703e62d585622aa1ac1"} Jul 15 23:48:38.880786 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.880684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"5077eca3740d23a087c7672dfb2655d1aa4d51e52f19c0085957a82128080c05"} Jul 15 23:48:38.880786 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.880731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"10056163b843783fd027b84c0518e94f50566bdd9cef84e81a1c7468cc699e69"} Jul 15 23:48:38.880786 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.880767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a91adafa-f20a-4bbf-8a7d-c6a110bddfb0","Type":"ContainerStarted","Data":"6f78404df33807d05baddd4ffc4dff945c1ddc6ff8b55e2520d682ea905419f9"} Jul 15 23:48:38.882180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.882147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-266qn" event={"ID":"5e664898-1dcd-4d36-9a2e-9fcc70c79725","Type":"ContainerStarted","Data":"69c44c78cf4f2e9d46f30241cbe1ee77f8e9e1897da8e406833a702ba7735c2c"} Jul 15 23:48:38.882297 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.882254 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:48:38.883492 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.883468 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxp97" event={"ID":"6ab8a1c8-57e8-47b4-999b-63f569069d7a","Type":"ContainerStarted","Data":"594646cd0ad58a5abf5f6e5b1e3a262031840b0ddd29f43a10edeed3e1f9e7c2"} Jul 15 23:48:38.918994 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.918942 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.9941861640000003 podStartE2EDuration="11.918925263s" totalImagesPullingTime="7.905703867s" totalInitContainerRuntime="19.035232ms" isStatefulPod=false podCreationTimestamp="2026-07-15 23:48:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:38.917400986 +0000 UTC m=+73.972731063" watchObservedRunningTime="2026-07-15 23:48:38.918925263 +0000 UTC m=+73.974255339" Jul 15 23:48:38.933630 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.933573 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-266qn" podStartSLOduration=67.716688299 podStartE2EDuration="1m13.933561878s" totalImagesPullingTime="6.216873579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:38.932809123 +0000 UTC m=+73.988139196" watchObservedRunningTime="2026-07-15 23:48:38.933561878 +0000 UTC m=+73.988891994" Jul 15 23:48:38.947918 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:38.947873 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xxp97" podStartSLOduration=66.701748341 podStartE2EDuration="1m12.947862435s" totalImagesPullingTime="6.246114094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 23:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:48:38.947174331 +0000 UTC m=+74.002504405" watchObservedRunningTime="2026-07-15 23:48:38.947862435 +0000 UTC m=+74.003192507" Jul 15 23:48:41.044463 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:41.044390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:41.044827 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:41.044497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:41.049397 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:41.049379 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:41.899265 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:41.899240 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:48:41.945972 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:41.945939 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:48:42.045239 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:42.045189 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:42.095253 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:42.095234 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:42.542046 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:42.542026 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:48:43.285478 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:43.285449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:45.622403 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:45.622379 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:45.622730 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:45.622416 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:48:49.270472 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:49.270362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:49.323902 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:49.323862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:48:50.880405 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:50.880372 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cfd9c65c8-tzsp7" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerName="console" containerID="cri-o://02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf" gracePeriod=15 Jul 15 23:48:51.650600 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.650573 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:51.761379 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761323 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvnwq\" (UniqueName: \"kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761379 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761373 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761565 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761399 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761565 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761418 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761565 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761451 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761709 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761585 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config\") pod \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\" (UID: \"1c1e4809-bc1b-4804-8b91-2197c98f04f4\") " Jul 15 23:48:51.761912 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761879 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:48:51.761912 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761878 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:48:51.762064 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.761963 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config" (OuterVolumeSpecName: "console-config") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:48:51.763758 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.763734 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:48:51.763963 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.763940 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:48:51.763963 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.763953 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq" (OuterVolumeSpecName: "kube-api-access-wvnwq") pod "1c1e4809-bc1b-4804-8b91-2197c98f04f4" (UID: "1c1e4809-bc1b-4804-8b91-2197c98f04f4"). InnerVolumeSpecName "kube-api-access-wvnwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 23:48:51.863111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863089 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.863111 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863108 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.863241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863119 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvnwq\" (UniqueName: \"kubernetes.io/projected/1c1e4809-bc1b-4804-8b91-2197c98f04f4-kube-api-access-wvnwq\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.863241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863127 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-oauth-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.863241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863135 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c1e4809-bc1b-4804-8b91-2197c98f04f4-console-oauth-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.863241 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.863143 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1e4809-bc1b-4804-8b91-2197c98f04f4-service-ca\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:48:51.937394 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.937367 2580 generic.go:356] "Generic (PLEG): container finished" podID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerID="02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf" exitCode=0 Jul 15 23:48:51.937751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.937405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfd9c65c8-tzsp7" event={"ID":"1c1e4809-bc1b-4804-8b91-2197c98f04f4","Type":"ContainerDied","Data":"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf"} Jul 15 23:48:51.937751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.937416 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfd9c65c8-tzsp7" Jul 15 23:48:51.937751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.937453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfd9c65c8-tzsp7" event={"ID":"1c1e4809-bc1b-4804-8b91-2197c98f04f4","Type":"ContainerDied","Data":"1e52400905021085abd06657819fd8e6c7930f8bd0ac007312b315f1e2ed0def"} Jul 15 23:48:51.937751 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.937475 2580 scope.go:117] "RemoveContainer" containerID="02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf" Jul 15 23:48:51.945673 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.945654 2580 scope.go:117] "RemoveContainer" containerID="02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf" Jul 15 23:48:51.945916 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:48:51.945899 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf\": container with ID starting with 02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf not found: ID does not exist" containerID="02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf" Jul 15 23:48:51.945968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.945924 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf"} err="failed to get container status \"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf\": rpc error: code = NotFound desc = could not find container \"02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf\": container with ID starting with 02ddce961bd779d074e9bff648e1793af8e2e8fb7408cb39fd6fa1393104f4cf not found: ID does not exist" Jul 15 23:48:51.960641 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.960459 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:51.963244 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:51.963206 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cfd9c65c8-tzsp7"] Jul 15 23:48:53.508548 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:53.508517 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" path="/var/lib/kubelet/pods/1c1e4809-bc1b-4804-8b91-2197c98f04f4/volumes" Jul 15 23:48:59.940933 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:48:59.940903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ps92_eca580ce-12b8-42b5-b4b6-1cba272266ed/serve-healthcheck-canary/0.log" Jul 15 23:49:05.626588 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:05.626558 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:49:05.630403 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:05.630378 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5fdcb84974-nknxh" Jul 15 23:49:06.966682 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:06.966648 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6876bfb45b-bn926" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerName="console" containerID="cri-o://05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07" gracePeriod=15 Jul 15 23:49:07.192948 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.192926 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:49:07.275775 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.275751 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.275883 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.275792 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c69rn\" (UniqueName: \"kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.275883 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.275833 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.275992 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.275904 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.276051 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276029 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.276112 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276087 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.276166 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276120 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle\") pod \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\" (UID: \"901b1679-4f2f-46bf-9b83-af87e88cbc8f\") " Jul 15 23:49:07.276595 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276349 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca" (OuterVolumeSpecName: "service-ca") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:49:07.276595 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276412 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-service-ca\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.276595 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276519 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:49:07.276595 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276537 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config" (OuterVolumeSpecName: "console-config") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:49:07.276861 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.276650 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:49:07.278081 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.278060 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn" (OuterVolumeSpecName: "kube-api-access-c69rn") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "kube-api-access-c69rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 23:49:07.278399 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.278372 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:49:07.278494 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.278418 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "901b1679-4f2f-46bf-9b83-af87e88cbc8f" (UID: "901b1679-4f2f-46bf-9b83-af87e88cbc8f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:49:07.377467 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377446 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c69rn\" (UniqueName: \"kubernetes.io/projected/901b1679-4f2f-46bf-9b83-af87e88cbc8f-kube-api-access-c69rn\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.377467 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377466 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-oauth-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.377591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377476 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-oauth-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.377591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377484 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.377591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377493 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/901b1679-4f2f-46bf-9b83-af87e88cbc8f-trusted-ca-bundle\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.377591 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.377501 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/901b1679-4f2f-46bf-9b83-af87e88cbc8f-console-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:49:07.998425 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.998397 2580 generic.go:356] "Generic (PLEG): container finished" podID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerID="05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07" exitCode=0 Jul 15 23:49:07.998736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.998441 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6876bfb45b-bn926" Jul 15 23:49:07.998736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.998459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6876bfb45b-bn926" event={"ID":"901b1679-4f2f-46bf-9b83-af87e88cbc8f","Type":"ContainerDied","Data":"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07"} Jul 15 23:49:07.998736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.998504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6876bfb45b-bn926" event={"ID":"901b1679-4f2f-46bf-9b83-af87e88cbc8f","Type":"ContainerDied","Data":"457e218bb1c0cf20f371859d4581f412c16f5ccee0d335ee5437dff0b9fd2207"} Jul 15 23:49:07.998736 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:07.998532 2580 scope.go:117] "RemoveContainer" containerID="05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07" Jul 15 23:49:08.006579 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:08.006562 2580 scope.go:117] "RemoveContainer" containerID="05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07" Jul 15 23:49:08.006862 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:49:08.006826 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07\": container with ID starting with 05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07 not found: ID does not exist" containerID="05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07" Jul 15 23:49:08.006933 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:08.006860 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07"} err="failed to get container status \"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07\": rpc error: code = NotFound desc = could not find container \"05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07\": container with ID starting with 05e8c7c4319e161bbdbb2ed073096fb1c7b5ec289c79e14aeacbb44e0409ec07 not found: ID does not exist" Jul 15 23:49:08.014557 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:08.014539 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:49:08.017555 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:08.017537 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6876bfb45b-bn926"] Jul 15 23:49:09.508260 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:09.508208 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" path="/var/lib/kubelet/pods/901b1679-4f2f-46bf-9b83-af87e88cbc8f/volumes" Jul 15 23:49:09.890862 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:09.890791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-266qn" Jul 15 23:49:11.176527 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:11.176497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:11.277448 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:11.277419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:11.327632 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:11.327603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:27.542464 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:27.542433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:49:27.561542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:27.561523 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:49:28.092874 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:28.092853 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 23:49:37.082796 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:37.082759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:37.136383 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:37.136340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:48.258950 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:48.258918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:48.308939 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:48.308912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:48.399423 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:48.399389 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:48.454923 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:48.454902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:49.001646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:49.001616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:49.104727 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:49.104695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:58.949072 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:58.949035 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:49:59.037746 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:59.037714 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:59.086161 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:59.086137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:59.137819 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:59.137800 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:49:59.239040 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:49:59.238984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:04.268732 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:04.268698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:04.369454 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:04.369423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:04.470383 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:04.470356 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:12.112494 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:12.112458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:23.969336 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:23.969198 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c465c66-f4884" podUID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" containerName="console" containerID="cri-o://f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77" gracePeriod=15 Jul 15 23:50:24.241209 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.241182 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:50:24.285262 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.285214 2580 generic.go:356] "Generic (PLEG): container finished" podID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" containerID="f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77" exitCode=0 Jul 15 23:50:24.285410 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.285275 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c465c66-f4884" Jul 15 23:50:24.285410 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.285303 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c465c66-f4884" event={"ID":"dd6381bf-5be8-42d9-b83f-692ce5e8a24b","Type":"ContainerDied","Data":"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77"} Jul 15 23:50:24.285410 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.285351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c465c66-f4884" event={"ID":"dd6381bf-5be8-42d9-b83f-692ce5e8a24b","Type":"ContainerDied","Data":"25997e771a97b1beb0eeb7f6a67f8567cbab37d6ec02677836a642dcb3dd92a0"} Jul 15 23:50:24.285410 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.285378 2580 scope.go:117] "RemoveContainer" containerID="f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77" Jul 15 23:50:24.293094 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.293076 2580 scope.go:117] "RemoveContainer" containerID="f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77" Jul 15 23:50:24.293377 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:50:24.293357 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77\": container with ID starting with f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77 not found: ID does not exist" containerID="f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77" Jul 15 23:50:24.293460 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.293387 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77"} err="failed to get container status \"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77\": rpc error: code = NotFound desc = could not find container \"f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77\": container with ID starting with f727a3bac9babb4db028c131c228331bc65851cc559a85f57a8608db0fe72a77 not found: ID does not exist" Jul 15 23:50:24.363677 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363653 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363770 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363686 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363770 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363723 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b62rg\" (UniqueName: \"kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363849 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363782 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363849 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363805 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363849 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363840 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.363977 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.363893 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert\") pod \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\" (UID: \"dd6381bf-5be8-42d9-b83f-692ce5e8a24b\") " Jul 15 23:50:24.364310 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.364282 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:50:24.364310 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.364295 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config" (OuterVolumeSpecName: "console-config") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:50:24.364310 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.364305 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca" (OuterVolumeSpecName: "service-ca") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:50:24.364517 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.364351 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 23:50:24.365912 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.365884 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:50:24.365993 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.365940 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 23:50:24.365993 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.365939 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg" (OuterVolumeSpecName: "kube-api-access-b62rg") pod "dd6381bf-5be8-42d9-b83f-692ce5e8a24b" (UID: "dd6381bf-5be8-42d9-b83f-692ce5e8a24b"). InnerVolumeSpecName "kube-api-access-b62rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 23:50:24.465094 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465075 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b62rg\" (UniqueName: \"kubernetes.io/projected/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-kube-api-access-b62rg\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465094 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465095 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-oauth-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465105 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-service-ca\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465113 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-trusted-ca-bundle\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465122 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-serving-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465130 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.465210 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.465139 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd6381bf-5be8-42d9-b83f-692ce5e8a24b-console-oauth-config\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 15 23:50:24.606126 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.606103 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:50:24.609360 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:24.609338 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c465c66-f4884"] Jul 15 23:50:25.508445 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:25.508411 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" path="/var/lib/kubelet/pods/dd6381bf-5be8-42d9-b83f-692ce5e8a24b/volumes" Jul 15 23:50:30.070399 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:30.070357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:32.465765 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:32.465727 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:32.575939 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:32.575911 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:50:47.885864 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:50:47.885829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:09.082360 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:09.082326 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:09.864465 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:09.864428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:12.694025 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:12.693988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:12.746919 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:12.746888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:13.600402 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:13.600367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:13.735833 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:13.735804 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:14.834601 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:14.834569 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:14.933377 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:14.933340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:15.174935 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.174865 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gld6n"] Jul 15 23:51:15.175144 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175133 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerName="console" Jul 15 23:51:15.175184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175145 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerName="console" Jul 15 23:51:15.175184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175158 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerName="console" Jul 15 23:51:15.175184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175164 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerName="console" Jul 15 23:51:15.175184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175174 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" containerName="console" Jul 15 23:51:15.175184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175179 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" containerName="console" Jul 15 23:51:15.175356 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175238 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c1e4809-bc1b-4804-8b91-2197c98f04f4" containerName="console" Jul 15 23:51:15.175356 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175248 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="901b1679-4f2f-46bf-9b83-af87e88cbc8f" containerName="console" Jul 15 23:51:15.175356 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.175256 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6381bf-5be8-42d9-b83f-692ce5e8a24b" containerName="console" Jul 15 23:51:15.178142 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.178123 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.181916 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.181900 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 23:51:15.183233 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.183201 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-blwql\"" Jul 15 23:51:15.183337 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.183203 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 23:51:15.199650 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.199628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gld6n"] Jul 15 23:51:15.240449 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.240426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.240545 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.240481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxh9t\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-kube-api-access-mxh9t\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.341113 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.341085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxh9t\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-kube-api-access-mxh9t\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.341202 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.341145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.350783 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.350759 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.350859 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.350819 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxh9t\" (UniqueName: \"kubernetes.io/projected/105c2b5d-c454-4167-ba7b-6f2b33e0a8ff-kube-api-access-mxh9t\") pod \"cert-manager-webhook-6486ffd997-gld6n\" (UID: \"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.495623 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.495605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:15.620949 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:15.620921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gld6n"] Jul 15 23:51:15.623966 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:51:15.623938 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod105c2b5d_c454_4167_ba7b_6f2b33e0a8ff.slice/crio-44604a21662fa7a5eeee4a33fc53dfccde6842128788406ca708d54733228d25 WatchSource:0}: Error finding container 44604a21662fa7a5eeee4a33fc53dfccde6842128788406ca708d54733228d25: Status 404 returned error can't find the container with id 44604a21662fa7a5eeee4a33fc53dfccde6842128788406ca708d54733228d25 Jul 15 23:51:16.473947 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:16.473909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" event={"ID":"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff","Type":"ContainerStarted","Data":"44604a21662fa7a5eeee4a33fc53dfccde6842128788406ca708d54733228d25"} Jul 15 23:51:17.231312 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:17.231263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:17.436169 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:17.436132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:19.486486 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:19.486454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" event={"ID":"105c2b5d-c454-4167-ba7b-6f2b33e0a8ff","Type":"ContainerStarted","Data":"0e9612339e5820636fda31bbc6e1d7c93206fcd6d8091b47e1d7b50b93d6d594"} Jul 15 23:51:19.486968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:19.486520 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:19.498122 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:19.498093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:19.503817 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:19.503772 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" podStartSLOduration=1.504438388 podStartE2EDuration="4.503753684s" totalImagesPullingTime="2.999315296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:51:19.503241645 +0000 UTC m=+234.558571719" watchObservedRunningTime="2026-07-15 23:51:19.503753684 +0000 UTC m=+234.559083758" Jul 15 23:51:19.598570 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:19.598544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:24.501855 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:24.501824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:24.605945 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:24.605909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:24.856303 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:24.856199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:25.492165 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:25.492139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gld6n" Jul 15 23:51:34.039909 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:34.039874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:34.140191 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:34.140161 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:36.238555 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:36.238524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:42.737298 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:42.737264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:42.838347 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:42.838312 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:47.890605 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:47.890533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:48.006420 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.006387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:48.514885 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.514853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:48.704737 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.704702 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:48.811953 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.811878 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:48.875146 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.875109 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk"] Jul 15 23:51:48.877444 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.877428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.887312 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.887296 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 23:51:48.887383 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.887306 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 23:51:48.887549 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.887532 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 23:51:48.887603 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.887578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-25q44\"" Jul 15 23:51:48.887645 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.887606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 23:51:48.900024 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.900000 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk"] Jul 15 23:51:48.913121 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.913099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.913214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.913131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-manifests\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.913214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.913155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-tmp\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.913214 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.913178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.913332 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.913251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4jbl\" (UniqueName: \"kubernetes.io/projected/f25211ed-8117-40e1-a947-012c37f2b045-kube-api-access-r4jbl\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:48.989662 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:48.989640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.014345 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-tmp\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.014437 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.014437 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4jbl\" (UniqueName: \"kubernetes.io/projected/f25211ed-8117-40e1-a947-012c37f2b045-kube-api-access-r4jbl\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.014577 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.014624 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-manifests\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.014935 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.014908 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-manifests\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.016592 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.016569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f25211ed-8117-40e1-a947-012c37f2b045-tmp\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.016800 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.016784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.016843 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.016799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f25211ed-8117-40e1-a947-012c37f2b045-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.022493 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.022473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4jbl\" (UniqueName: \"kubernetes.io/projected/f25211ed-8117-40e1-a947-012c37f2b045-kube-api-access-r4jbl\") pod \"opendatahub-operator-controller-manager-7b4446c48-d59pk\" (UID: \"f25211ed-8117-40e1-a947-012c37f2b045\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.090387 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.090336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.187468 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.187445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:49.191405 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.191380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.292209 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.292180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.320137 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.320112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk"] Jul 15 23:51:49.322893 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:51:49.322870 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25211ed_8117_40e1_a947_012c37f2b045.slice/crio-fc6a7ffe5b138017e4eaacd6de23d91520cbc67afaa3033a61d2f8d1fcf89eb7 WatchSource:0}: Error finding container fc6a7ffe5b138017e4eaacd6de23d91520cbc67afaa3033a61d2f8d1fcf89eb7: Status 404 returned error can't find the container with id fc6a7ffe5b138017e4eaacd6de23d91520cbc67afaa3033a61d2f8d1fcf89eb7 Jul 15 23:51:49.398538 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.398484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.502940 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.502915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:51:49.592728 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:49.592702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" event={"ID":"f25211ed-8117-40e1-a947-012c37f2b045","Type":"ContainerStarted","Data":"fc6a7ffe5b138017e4eaacd6de23d91520cbc67afaa3033a61d2f8d1fcf89eb7"} Jul 15 23:51:52.605959 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:52.605875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" event={"ID":"f25211ed-8117-40e1-a947-012c37f2b045","Type":"ContainerStarted","Data":"c7b5d728848e378cdfbf0937d70eb1617251294cf8729e3bff9d48b7d962d908"} Jul 15 23:51:53.611109 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:53.611030 2580 generic.go:356] "Generic (PLEG): container finished" podID="f25211ed-8117-40e1-a947-012c37f2b045" containerID="c7b5d728848e378cdfbf0937d70eb1617251294cf8729e3bff9d48b7d962d908" exitCode=0 Jul 15 23:51:53.611109 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:53.611082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" event={"ID":"f25211ed-8117-40e1-a947-012c37f2b045","Type":"ContainerDied","Data":"c7b5d728848e378cdfbf0937d70eb1617251294cf8729e3bff9d48b7d962d908"} Jul 15 23:51:54.617047 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:54.617013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" event={"ID":"f25211ed-8117-40e1-a947-012c37f2b045","Type":"ContainerStarted","Data":"a695f681802dfc8de787045c05719bfa47fd7bcc6df113b3b157e45ed1e4a843"} Jul 15 23:51:54.617426 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:54.617149 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:51:54.640520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:51:54.640475 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" podStartSLOduration=2.310483164 podStartE2EDuration="6.640460778s" totalImagesPullingTime="3.313780702s" totalInitContainerRuntime="1.016196912s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:51:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:51:54.639705671 +0000 UTC m=+269.695035764" watchObservedRunningTime="2026-07-15 23:51:54.640460778 +0000 UTC m=+269.695790851" Jul 15 23:52:02.536542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:02.536506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:03.960725 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:03.960691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:05.622862 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:05.622831 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4446c48-d59pk" Jul 15 23:52:07.397138 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.397098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:07.521008 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.520965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:07.801135 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.801101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:07.904024 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.903992 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-797ddd4965-75pss"] Jul 15 23:52:07.913559 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.913529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:07.917520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.917492 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 23:52:07.917520 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.917515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 23:52:07.918733 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.918714 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 23:52:07.918733 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.918730 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qkhkj\"" Jul 15 23:52:07.918861 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.918718 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 23:52:07.927768 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.927738 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797ddd4965-75pss"] Jul 15 23:52:07.967434 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.967395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:07.975588 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.975562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4131bbca-92b1-4bc4-b3d4-323f0332c887-tmp\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:07.975728 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.975626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4131bbca-92b1-4bc4-b3d4-323f0332c887-tls-certs\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:07.975728 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:07.975650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gklkz\" (UniqueName: \"kubernetes.io/projected/4131bbca-92b1-4bc4-b3d4-323f0332c887-kube-api-access-gklkz\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.076542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.076440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4131bbca-92b1-4bc4-b3d4-323f0332c887-tls-certs\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.076542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.076478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gklkz\" (UniqueName: \"kubernetes.io/projected/4131bbca-92b1-4bc4-b3d4-323f0332c887-kube-api-access-gklkz\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.076542 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.076530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4131bbca-92b1-4bc4-b3d4-323f0332c887-tmp\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.078929 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.078899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4131bbca-92b1-4bc4-b3d4-323f0332c887-tmp\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.079199 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.079180 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4131bbca-92b1-4bc4-b3d4-323f0332c887-tls-certs\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.095658 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.095624 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gklkz\" (UniqueName: \"kubernetes.io/projected/4131bbca-92b1-4bc4-b3d4-323f0332c887-kube-api-access-gklkz\") pod \"kube-auth-proxy-797ddd4965-75pss\" (UID: \"4131bbca-92b1-4bc4-b3d4-323f0332c887\") " pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.188974 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.188929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:08.223324 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.223288 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" Jul 15 23:52:08.285173 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.285069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:08.354481 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.354405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:08.373882 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.373851 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797ddd4965-75pss"] Jul 15 23:52:08.377056 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:52:08.377028 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4131bbca_92b1_4bc4_b3d4_323f0332c887.slice/crio-a20438830b2c680ebebca5c13f40d020978a75f6ba9124fc20dcfba01089675b WatchSource:0}: Error finding container a20438830b2c680ebebca5c13f40d020978a75f6ba9124fc20dcfba01089675b: Status 404 returned error can't find the container with id a20438830b2c680ebebca5c13f40d020978a75f6ba9124fc20dcfba01089675b Jul 15 23:52:08.423908 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.423868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:08.672073 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.671986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" event={"ID":"4131bbca-92b1-4bc4-b3d4-323f0332c887","Type":"ContainerStarted","Data":"a20438830b2c680ebebca5c13f40d020978a75f6ba9124fc20dcfba01089675b"} Jul 15 23:52:08.924691 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.924615 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4gjr"] Jul 15 23:52:08.927577 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.927555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:08.928799 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.928773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:08.930179 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.930159 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 23:52:08.930393 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.930377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4prv9\"" Jul 15 23:52:08.938863 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.938840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4gjr"] Jul 15 23:52:08.983981 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.983934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:08.984209 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:08.984052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qptj7\" (UniqueName: \"kubernetes.io/projected/0433cea8-009b-4694-9f3b-68da5b267357-kube-api-access-qptj7\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.080871 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.080832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:09.085086 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.085049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qptj7\" (UniqueName: \"kubernetes.io/projected/0433cea8-009b-4694-9f3b-68da5b267357-kube-api-access-qptj7\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.085323 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.085299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.085468 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:52:09.085448 2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 23:52:09.085540 ip-10-0-129-211 kubenswrapper[2580]: E0715 23:52:09.085527 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs podName:0433cea8-009b-4694-9f3b-68da5b267357 nodeName:}" failed. No retries permitted until 2026-07-15 23:52:09.585506412 +0000 UTC m=+284.640836466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs") pod "model-serving-api-54d89bd5fc-t4gjr" (UID: "0433cea8-009b-4694-9f3b-68da5b267357") : secret "model-serving-api-tls" not found Jul 15 23:52:09.104153 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.104106 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qptj7\" (UniqueName: \"kubernetes.io/projected/0433cea8-009b-4694-9f3b-68da5b267357-kube-api-access-qptj7\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.153999 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.153960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:09.389034 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.388991 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:09.590145 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.590109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.592712 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.592684 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0433cea8-009b-4694-9f3b-68da5b267357-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4gjr\" (UID: \"0433cea8-009b-4694-9f3b-68da5b267357\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.839573 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.839529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:09.987274 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.987197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4gjr"] Jul 15 23:52:09.988422 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:09.988384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:09.989574 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:52:09.989542 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0433cea8_009b_4694_9f3b_68da5b267357.slice/crio-ab917b53e42a0b89fc41f43bc630aa09f21100b44c2f50f26cb5fd87cdc65268 WatchSource:0}: Error finding container ab917b53e42a0b89fc41f43bc630aa09f21100b44c2f50f26cb5fd87cdc65268: Status 404 returned error can't find the container with id ab917b53e42a0b89fc41f43bc630aa09f21100b44c2f50f26cb5fd87cdc65268 Jul 15 23:52:10.681876 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:10.681834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" event={"ID":"0433cea8-009b-4694-9f3b-68da5b267357","Type":"ContainerStarted","Data":"ab917b53e42a0b89fc41f43bc630aa09f21100b44c2f50f26cb5fd87cdc65268"} Jul 15 23:52:11.692734 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:11.692695 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" event={"ID":"4131bbca-92b1-4bc4-b3d4-323f0332c887","Type":"ContainerStarted","Data":"c5b4cb93c1a9e4bf15f25030aea90f688c385ab70f963b087266f7994fda8978"} Jul 15 23:52:11.710562 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:11.710503 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-797ddd4965-75pss" podStartSLOduration=2.070274794 podStartE2EDuration="4.710486083s" totalImagesPullingTime="2.640211289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:52:11.709327781 +0000 UTC m=+286.764657854" watchObservedRunningTime="2026-07-15 23:52:11.710486083 +0000 UTC m=+286.765816182" Jul 15 23:52:12.697711 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:12.697605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" event={"ID":"0433cea8-009b-4694-9f3b-68da5b267357","Type":"ContainerStarted","Data":"e41726b4037d4b30b8a8e16ddf911e63dbc02a08c5a7efd05f575d30ed4a7ce7"} Jul 15 23:52:12.712971 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:12.712913 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" podStartSLOduration=2.411762606 podStartE2EDuration="4.712897076s" totalImagesPullingTime="2.30113447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:52:12.71209188 +0000 UTC m=+287.767421954" watchObservedRunningTime="2026-07-15 23:52:12.712897076 +0000 UTC m=+287.768227185" Jul 15 23:52:13.701776 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:13.701743 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:14.112166 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.112126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:14.169208 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.169174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:14.677683 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.677650 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx"] Jul 15 23:52:14.681047 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.681029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.683906 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.683878 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 23:52:14.684044 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.683958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xfdvf\"" Jul 15 23:52:14.692614 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.692579 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx"] Jul 15 23:52:14.704544 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.704508 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:14.736482 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736704 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736704 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736704 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736861 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736726 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736861 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5gfd\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-kube-api-access-m5gfd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736861 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736964 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.736964 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.736938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838358 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5gfd\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-kube-api-access-m5gfd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838569 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838822 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838822 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838822 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838958 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.838958 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.838938 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.839061 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.839036 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.839117 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.839079 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.839522 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.839497 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.840994 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.840975 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.841436 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.841415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.850684 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.850657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.850847 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.850679 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5gfd\" (UniqueName: \"kubernetes.io/projected/e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f-kube-api-access-m5gfd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ptrzx\" (UID: \"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:14.916212 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.916181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:14.992629 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:14.992599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:15.016197 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:15.016158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:15.136674 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:15.136650 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx"] Jul 15 23:52:15.138365 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:52:15.138344 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58a1ce4_d2d3_4e7e_8b69_f5d3acdf0c4f.slice/crio-529444d70cd91f79593f3f0de4d62293dc4724fec78ce9936fe18595b840a2c9 WatchSource:0}: Error finding container 529444d70cd91f79593f3f0de4d62293dc4724fec78ce9936fe18595b840a2c9: Status 404 returned error can't find the container with id 529444d70cd91f79593f3f0de4d62293dc4724fec78ce9936fe18595b840a2c9 Jul 15 23:52:15.717184 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:15.717139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" event={"ID":"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f","Type":"ContainerStarted","Data":"529444d70cd91f79593f3f0de4d62293dc4724fec78ce9936fe18595b840a2c9"} Jul 15 23:52:16.507843 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.507805 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx"] Jul 15 23:52:16.511895 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.511865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.514878 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.514759 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7vcll\"" Jul 15 23:52:16.514878 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.514818 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 23:52:16.515104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.514893 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 23:52:16.515104 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.515068 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 23:52:16.523843 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.523811 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx"] Jul 15 23:52:16.656879 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.656846 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4509e29d-4571-423e-9129-1d1fb97a614a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.656895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.656945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvp5z\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-kube-api-access-mvp5z\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.656974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657035 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.657001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657196 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.657072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.657196 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.657111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758457 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758457 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4509e29d-4571-423e-9129-1d1fb97a614a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvp5z\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-kube-api-access-mvp5z\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.758968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.758627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.759471 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.759425 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.761485 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.761461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4509e29d-4571-423e-9129-1d1fb97a614a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.761684 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.761660 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.761741 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.761666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.761741 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.761720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4509e29d-4571-423e-9129-1d1fb97a614a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.770768 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.770694 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.770768 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.770718 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvp5z\" (UniqueName: \"kubernetes.io/projected/4509e29d-4571-423e-9129-1d1fb97a614a-kube-api-access-mvp5z\") pod \"istiod-openshift-gateway-94bb8fbfd-44qlx\" (UID: \"4509e29d-4571-423e-9129-1d1fb97a614a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.825753 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.825710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:16.922513 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.922480 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-755cb7796d-7btms"] Jul 15 23:52:16.926009 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.925981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:16.929198 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.929164 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 23:52:16.929694 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.929674 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 23:52:16.930514 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.930493 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 23:52:16.930804 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.930786 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rvx4t\"" Jul 15 23:52:16.931261 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.931209 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 23:52:16.931432 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.931413 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 23:52:16.957275 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:16.957233 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-755cb7796d-7btms"] Jul 15 23:52:17.061850 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.061759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl57k\" (UniqueName: \"kubernetes.io/projected/18134957-5a78-40aa-9f86-6a37cebe12c5-kube-api-access-rl57k\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.061850 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.061825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.062091 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.061876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-metrics-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.062091 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.061926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18134957-5a78-40aa-9f86-6a37cebe12c5-manager-config\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.163566 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.163522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-metrics-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.163796 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.163584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18134957-5a78-40aa-9f86-6a37cebe12c5-manager-config\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.163796 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.163653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl57k\" (UniqueName: \"kubernetes.io/projected/18134957-5a78-40aa-9f86-6a37cebe12c5-kube-api-access-rl57k\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.163796 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.163698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.164749 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.164720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18134957-5a78-40aa-9f86-6a37cebe12c5-manager-config\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.167563 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.167507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-metrics-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.168178 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.168149 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18134957-5a78-40aa-9f86-6a37cebe12c5-cert\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.173665 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.173634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl57k\" (UniqueName: \"kubernetes.io/projected/18134957-5a78-40aa-9f86-6a37cebe12c5-kube-api-access-rl57k\") pod \"lws-controller-manager-755cb7796d-7btms\" (UID: \"18134957-5a78-40aa-9f86-6a37cebe12c5\") " pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.239321 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.239265 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:17.770890 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.770330 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 23:52:17.770890 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.770430 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 23:52:17.770890 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.770480 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 23:52:17.874439 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.874414 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-755cb7796d-7btms"] Jul 15 23:52:17.877038 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:52:17.877003 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18134957_5a78_40aa_9f86_6a37cebe12c5.slice/crio-9b39cfccd66994cafcceb14cab717ee3833f460d0c0ceb5db1b4eeed0e66f2e7 WatchSource:0}: Error finding container 9b39cfccd66994cafcceb14cab717ee3833f460d0c0ceb5db1b4eeed0e66f2e7: Status 404 returned error can't find the container with id 9b39cfccd66994cafcceb14cab717ee3833f460d0c0ceb5db1b4eeed0e66f2e7 Jul 15 23:52:17.887773 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:17.887742 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx"] Jul 15 23:52:17.890502 ip-10-0-129-211 kubenswrapper[2580]: W0715 23:52:17.890474 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4509e29d_4571_423e_9129_1d1fb97a614a.slice/crio-3a13373460d25db09d145d7c49fe86563e00c9900d896313d4dc5a0b458ec510 WatchSource:0}: Error finding container 3a13373460d25db09d145d7c49fe86563e00c9900d896313d4dc5a0b458ec510: Status 404 returned error can't find the container with id 3a13373460d25db09d145d7c49fe86563e00c9900d896313d4dc5a0b458ec510 Jul 15 23:52:18.735570 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:18.735524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" event={"ID":"18134957-5a78-40aa-9f86-6a37cebe12c5","Type":"ContainerStarted","Data":"9b39cfccd66994cafcceb14cab717ee3833f460d0c0ceb5db1b4eeed0e66f2e7"} Jul 15 23:52:18.737617 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:18.737583 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" event={"ID":"4509e29d-4571-423e-9129-1d1fb97a614a","Type":"ContainerStarted","Data":"3a13373460d25db09d145d7c49fe86563e00c9900d896313d4dc5a0b458ec510"} Jul 15 23:52:18.739804 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:18.739770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" event={"ID":"e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f","Type":"ContainerStarted","Data":"0b99d5633907768060ef45752c970a812094bcc5aa013786e67de273b6930ca0"} Jul 15 23:52:18.811878 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:18.811811 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" podStartSLOduration=2.182214202 podStartE2EDuration="4.81178347s" totalImagesPullingTime="2.629569268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:52:18.809214984 +0000 UTC m=+293.864545072" watchObservedRunningTime="2026-07-15 23:52:18.81178347 +0000 UTC m=+293.867113544" Jul 15 23:52:18.993118 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:18.993080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:19.003169 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.003116 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:19.236585 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.236549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:19.404605 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.404511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:19.515459 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.515419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:19.744788 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.744751 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:19.745866 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:19.745843 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ptrzx" Jul 15 23:52:20.538759 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.538725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:20.637197 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.637094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:20.702824 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.702794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:20.750480 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.750439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" event={"ID":"18134957-5a78-40aa-9f86-6a37cebe12c5","Type":"ContainerStarted","Data":"cd8f053a76fa52413f23802552424cc38fb26763987d806bcb39025f5edd0955"} Jul 15 23:52:20.750646 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.750628 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:52:20.767315 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.767257 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" podStartSLOduration=2.355051177 podStartE2EDuration="4.767215575s" totalImagesPullingTime="2.412164398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:52:20.766686923 +0000 UTC m=+295.822016996" watchObservedRunningTime="2026-07-15 23:52:20.767215575 +0000 UTC m=+295.822545649" Jul 15 23:52:20.877442 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:20.877407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:52:24.711560 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:24.711529 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-t4gjr" Jul 15 23:52:25.338881 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:25.338855 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 23:52:26.313968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:26.313376 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 23:52:26.313968 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:26.313441 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 23:52:26.781288 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:26.781254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" event={"ID":"4509e29d-4571-423e-9129-1d1fb97a614a","Type":"ContainerStarted","Data":"8359b716cdc425891fa13a8f9dc0fbb6a73a2176662866f10c2ee6723a212a53"} Jul 15 23:52:26.781500 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:26.781374 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:26.828131 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:26.828062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" podStartSLOduration=2.408242325 podStartE2EDuration="10.828027776s" totalImagesPullingTime="8.419785451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 23:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 23:52:26.827418107 +0000 UTC m=+301.882748221" watchObservedRunningTime="2026-07-15 23:52:26.828027776 +0000 UTC m=+301.883357851" Jul 15 23:52:27.788392 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:27.788366 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44qlx" Jul 15 23:52:31.755950 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:52:31.755917 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-755cb7796d-7btms" Jul 15 23:53:08.827502 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:08.827467 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:08.924865 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:08.924833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:09.026648 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:09.026615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:09.756541 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:09.756505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:09.949538 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:09.949495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:35.319471 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:35.319380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:35.419666 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:35.419634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:37.287663 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:37.287623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:37.714730 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:37.714634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:38.290208 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:38.290179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:38.390519 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:38.390484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:38.490622 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:38.490590 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:39.009471 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:39.009442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:39.107349 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:39.107322 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:41.547694 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:41.547648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:42.915930 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:42.915886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:43.316064 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:43.316031 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:44.347519 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:44.347486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:44.450750 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:44.450719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:45.926680 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:45.926648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:46.025494 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:46.025458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:48.118305 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:48.118265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:49.166155 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:49.166118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:49.274770 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:49.274739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:49.453201 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:49.453121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:49.554055 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:49.554025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:49.953541 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:49.953506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:51.596868 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:51.596837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:51.722455 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:51.722417 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:53:51.816801 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:53:51.816769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:00.924238 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:00.924193 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:01.024330 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:01.024294 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:03.373558 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:03.373527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:03.474086 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:03.474052 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:08.129243 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:08.127166 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:08.226848 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:08.226805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:11.927888 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:11.927845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:12.025705 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:12.025668 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:15.068939 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:15.068902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:15.168518 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:15.168470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:20.658516 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:20.658480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:20.758491 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:20.758452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:23.106999 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:23.106965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:23.164351 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:23.164308 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:27.854713 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:27.854675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:27.956197 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:27.956162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:29.481213 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:29.481178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:29.587440 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:29.587411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:32.053461 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:32.053427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:32.104701 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:32.104668 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:40.868609 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:40.868577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:40.968033 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:40.967987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:41.576588 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:41.576558 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:41.634136 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:41.634086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:43.308961 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:43.308930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:43.367280 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:43.367247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:46.311422 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:46.311377 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:46.423450 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:46.423421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:46.524001 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:46.523971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:47.133034 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:47.132961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:47.207699 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:47.207666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:48.069783 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:48.069706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:48.174673 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:48.174637 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:54:49.011792 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:54:49.011759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:02.386173 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:02.386138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:02.485381 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:02.485347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:02.834886 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:02.834838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:14.835012 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:14.834981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:15.045650 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:15.045611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:24.196813 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:24.196772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:24.293841 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:24.293805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:24.357009 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:24.356971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:24.990137 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:24.990102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:28.298749 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:28.298710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:28.396784 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:28.396749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:58.903323 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:58.903286 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:59.002592 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:59.002559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:59.433599 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:59.433557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:59.788059 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:59.788029 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:55:59.948245 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:55:59.948204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:00.042344 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:00.042170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:09.791471 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:09.791443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:09.894528 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:09.894490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:18.743990 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:18.743905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:18.845260 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:18.845211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:24.319076 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:24.319043 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:28.790869 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:28.790833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:34.755824 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:34.755794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:34.856180 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:34.856147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:56:55.328093 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:56:55.328056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:04.030457 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:04.030414 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:17.951007 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:17.950973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:21.334200 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:21.334161 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:28.989809 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:28.989773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:30.302747 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:30.302713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:39.021123 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:39.021088 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:57:45.575652 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:57:45.575618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:59:05.412260 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:59:05.412204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:59:05.506775 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:59:05.506743 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:59:06.493109 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:59:06.493081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 15 23:59:06.592732 ip-10-0-129-211 kubenswrapper[2580]: I0715 23:59:06.592704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:00:16.588878 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:00:16.587299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:00:17.914823 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:00:17.914785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:00:31.404322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:00:31.404289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:01.956555 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:01.956519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:02.057756 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:02.057726 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:07.414964 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:07.414916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:07.474195 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:07.474170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:07.553040 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:07.553013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:07.753668 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:07.753643 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:13.471981 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:13.471945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:01:13.540362 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:01:13.540340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:02:18.155781 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:02:18.155710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:03:35.799886 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:03:35.799853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:09:19.473160 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:09:19.473122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:09:20.907093 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:09:20.907044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:10:08.910707 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:10:08.910670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:10:08.962618 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:10:08.962585 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:10.228525 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:10.228488 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:19.362184 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:19.362150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:27.635936 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:27.635900 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:32.744248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:32.744194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:33.366120 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:33.366092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:35.854410 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:35.854380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:35.953240 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:35.953199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:36.308209 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:36.308158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:41.991082 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:41.991022 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.047558 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.047533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.155155 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.155127 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:42.158800 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.158780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.161679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.161660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f093c30e-openshift-default-dockercfg-vx4cl\"" Jul 16 00:12:42.180503 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.180482 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:42.192393 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.192374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.268740 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.268723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.318914 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.318887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319025 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.318925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319025 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.318997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319139 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319139 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319103 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319139 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qfm4\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.319314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.319301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.365024 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.364999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.420200 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qfm4\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420898 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420852 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420988 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420904 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420988 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.420988 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.420953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.421109 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.421057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.422732 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.422714 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.422970 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.422952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.428728 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.428706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qfm4\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.428829 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.428808 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token\") pod \"e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.469564 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.469536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:42.548291 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.548259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:42.610867 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.610809 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:42.614045 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:12:42.614019 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c4dc4d_d238_4d96_9278_fd01bb3b6ec1.slice/crio-b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1 WatchSource:0}: Error finding container b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1: Status 404 returned error can't find the container with id b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1 Jul 16 00:12:42.616107 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.616078 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:42.616177 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.616140 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:42.616177 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.616166 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:42.932375 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:42.932299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:43.032536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.032499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:43.079880 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.079846 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" event={"ID":"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1","Type":"ContainerStarted","Data":"8b9a0c89ebb30c06d050d3da4a24bba884f5b4fbe0ebe28aa13805eac228266a"} Jul 16 00:12:43.080000 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.079900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" event={"ID":"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1","Type":"ContainerStarted","Data":"b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1"} Jul 16 00:12:43.104333 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.104291 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" podStartSLOduration=1.104275452 podStartE2EDuration="1.104275452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:12:43.103268273 +0000 UTC m=+1518.158598365" watchObservedRunningTime="2026-07-16 00:12:43.104275452 +0000 UTC m=+1518.159605524" Jul 16 00:12:43.109026 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.109005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:43.470131 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.470098 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:43.474749 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:43.474725 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:44.084297 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:44.084268 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:44.085068 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:44.085050 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:45.973131 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:45.973104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:48.493562 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:48.493472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:48.493987 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:48.493729 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" podUID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" containerName="istio-proxy" containerID="cri-o://8b9a0c89ebb30c06d050d3da4a24bba884f5b4fbe0ebe28aa13805eac228266a" gracePeriod=30 Jul 16 00:12:48.523397 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:48.523368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:48.617993 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:48.617961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:50.584562 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:50.584530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:53.127828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.127771 2580 generic.go:356] "Generic (PLEG): container finished" podID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" containerID="8b9a0c89ebb30c06d050d3da4a24bba884f5b4fbe0ebe28aa13805eac228266a" exitCode=0 Jul 16 00:12:53.128151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.127838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" event={"ID":"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1","Type":"ContainerDied","Data":"8b9a0c89ebb30c06d050d3da4a24bba884f5b4fbe0ebe28aa13805eac228266a"} Jul 16 00:12:53.128151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.127890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" event={"ID":"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1","Type":"ContainerDied","Data":"b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1"} Jul 16 00:12:53.128151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.127908 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1a0e34a669fdd87eff5ca3f9d09bcbb854cae0c4f333fed6be8a094d5a366d1" Jul 16 00:12:53.136773 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.136746 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:53.319411 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319377 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319418 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319470 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319500 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319568 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319796 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319599 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319796 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319624 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qfm4\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319796 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319666 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.319796 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.319718 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert\") pod \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\" (UID: \"61c4dc4d-d238-4d96-9278-fd01bb3b6ec1\") " Jul 16 00:12:53.320125 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.320092 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:12:53.320320 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.320292 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:12:53.320408 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.320301 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data" (OuterVolumeSpecName: "istio-data") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:12:53.320453 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.320427 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:12:53.320491 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.320448 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:12:53.322041 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.322019 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4" (OuterVolumeSpecName: "kube-api-access-9qfm4") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "kube-api-access-9qfm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:12:53.322142 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.322038 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token" (OuterVolumeSpecName: "istio-token") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:12:53.322195 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.322145 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:12:53.322195 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.322142 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" (UID: "61c4dc4d-d238-4d96-9278-fd01bb3b6ec1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:12:53.421051 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421028 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421051 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421047 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421056 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421065 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421074 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qfm4\" (UniqueName: \"kubernetes.io/projected/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-kube-api-access-9qfm4\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421100 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421108 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421116 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:53.421182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:53.421123 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:12:54.132470 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.132438 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf" Jul 16 00:12:54.153326 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.153299 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:54.157187 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.157164 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f093c30e-openshift-default-7775749cf7-dnqtf"] Jul 16 00:12:54.345753 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.345722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.445957 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.445888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.602854 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.602821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.698558 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.698497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.746994 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.746962 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:12:54.747533 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.747511 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" containerName="istio-proxy" Jul 16 00:12:54.747533 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.747532 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" containerName="istio-proxy" Jul 16 00:12:54.747695 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.747609 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" containerName="istio-proxy" Jul 16 00:12:54.750887 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.750865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.753936 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.753912 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-447d0dec-gw-openshift-default-dockercfg-n82kx\"" Jul 16 00:12:54.765932 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.765912 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:12:54.790982 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.790958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.831569 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vnk2\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.831849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.832069 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.831856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.891028 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.891001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:54.932254 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932372 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vnk2\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932372 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932372 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932862 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932839 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.932942 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932892 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.933000 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.932942 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.933073 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.933051 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.933125 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.933060 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.934809 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.934792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.935105 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.935089 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.939728 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.939708 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:54.939728 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:54.939722 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vnk2\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2\") pod \"e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:55.061619 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.061599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:55.077754 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.077735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:55.185885 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.185858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:55.200995 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.200973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:12:55.202951 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:12:55.202928 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89803e4_e640_4128_8b7c_3d5c5fdacb58.slice/crio-fd302d733f93601b080dcd9d62bc2c94b89a88806d734020ae42f1acdd065781 WatchSource:0}: Error finding container fd302d733f93601b080dcd9d62bc2c94b89a88806d734020ae42f1acdd065781: Status 404 returned error can't find the container with id fd302d733f93601b080dcd9d62bc2c94b89a88806d734020ae42f1acdd065781 Jul 16 00:12:55.205135 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.205109 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:55.205258 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.205167 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:55.205258 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.205194 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:12:55.314368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.314298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:55.509139 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.509111 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61c4dc4d-d238-4d96-9278-fd01bb3b6ec1" path="/var/lib/kubelet/pods/61c4dc4d-d238-4d96-9278-fd01bb3b6ec1/volumes" Jul 16 00:12:55.679507 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.679423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:55.785053 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:55.785024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:12:56.145285 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:56.145254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" event={"ID":"a89803e4-e640-4128-8b7c-3d5c5fdacb58","Type":"ContainerStarted","Data":"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687"} Jul 16 00:12:56.145449 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:56.145309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" event={"ID":"a89803e4-e640-4128-8b7c-3d5c5fdacb58","Type":"ContainerStarted","Data":"fd302d733f93601b080dcd9d62bc2c94b89a88806d734020ae42f1acdd065781"} Jul 16 00:12:56.164461 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:56.164421 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" podStartSLOduration=2.164405716 podStartE2EDuration="2.164405716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:12:56.162668228 +0000 UTC m=+1531.217998300" watchObservedRunningTime="2026-07-16 00:12:56.164405716 +0000 UTC m=+1531.219735790" Jul 16 00:12:57.062676 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:57.062644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:57.067586 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:57.067560 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:57.149522 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:57.149497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:57.150304 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:57.150287 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:12:58.028989 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:12:58.028957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:00.627341 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:00.627299 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:13:00.627783 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:00.627628 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" podUID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" containerName="istio-proxy" containerID="cri-o://677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687" gracePeriod=30 Jul 16 00:13:00.650301 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:00.650270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:00.754120 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:00.754077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:02.212821 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:02.212793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.412838 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.412800 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.480914 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.480893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.633821 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.633792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.687982 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.687959 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:13:06.718601 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.718527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.729617 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729595 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vnk2\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729747 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729630 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729747 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729661 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729747 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729695 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729747 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729723 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729953 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729802 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729953 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729872 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729953 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729902 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.729953 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729921 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:13:06.729953 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.729937 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs\") pod \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\" (UID: \"a89803e4-e640-4128-8b7c-3d5c5fdacb58\") " Jul 16 00:13:06.730300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.730244 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.730300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.730244 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:13:06.730448 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.730328 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:13:06.730505 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.730448 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:13:06.730819 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.730773 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data" (OuterVolumeSpecName: "istio-data") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:13:06.731961 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.731937 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2" (OuterVolumeSpecName: "kube-api-access-2vnk2") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "kube-api-access-2vnk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:13:06.732110 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.732092 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:13:06.732305 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.732286 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:13:06.732454 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.732441 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token" (OuterVolumeSpecName: "istio-token") pod "a89803e4-e640-4128-8b7c-3d5c5fdacb58" (UID: "a89803e4-e640-4128-8b7c-3d5c5fdacb58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:13:06.803068 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.803045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.831081 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831058 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831081 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831080 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831090 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vnk2\" (UniqueName: \"kubernetes.io/projected/a89803e4-e640-4128-8b7c-3d5c5fdacb58-kube-api-access-2vnk2\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831101 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831110 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831118 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831129 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.831248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.831140 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89803e4-e640-4128-8b7c-3d5c5fdacb58-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:13:06.861786 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.861763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:06.980084 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:06.980010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:07.194601 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.194570 2580 generic.go:356] "Generic (PLEG): container finished" podID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" containerID="677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687" exitCode=0 Jul 16 00:13:07.194762 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.194627 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" Jul 16 00:13:07.194762 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.194663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" event={"ID":"a89803e4-e640-4128-8b7c-3d5c5fdacb58","Type":"ContainerDied","Data":"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687"} Jul 16 00:13:07.194762 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.194756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj" event={"ID":"a89803e4-e640-4128-8b7c-3d5c5fdacb58","Type":"ContainerDied","Data":"fd302d733f93601b080dcd9d62bc2c94b89a88806d734020ae42f1acdd065781"} Jul 16 00:13:07.194993 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.194789 2580 scope.go:117] "RemoveContainer" containerID="677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687" Jul 16 00:13:07.204052 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.204027 2580 scope.go:117] "RemoveContainer" containerID="677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687" Jul 16 00:13:07.204358 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:13:07.204339 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687\": container with ID starting with 677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687 not found: ID does not exist" containerID="677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687" Jul 16 00:13:07.204415 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.204364 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687"} err="failed to get container status \"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687\": rpc error: code = NotFound desc = could not find container \"677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687\": container with ID starting with 677eda09e3fda7278acb0becf7547260011e0a0847a75224ea4cbd5362ede687 not found: ID does not exist" Jul 16 00:13:07.216083 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.216058 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:13:07.219815 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.219787 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-447d0dec-gw-openshift-default-b954c76bc-nj9gj"] Jul 16 00:13:07.372323 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.372295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:07.476378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.476357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:07.509264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:07.509212 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" path="/var/lib/kubelet/pods/a89803e4-e640-4128-8b7c-3d5c5fdacb58/volumes" Jul 16 00:13:09.931448 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:09.931421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:12.261957 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:12.261921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:12.370918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:12.370890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:14.125655 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:14.125627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:18.046546 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:18.046514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:18.148009 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:18.147983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:18.522968 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:18.522940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:18.621257 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:18.621203 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:18.721418 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:18.721392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:20.132980 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:20.132950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:26.015391 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:26.015362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:26.117903 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:26.117880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:26.216379 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:26.216357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:26.316671 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:26.316611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:26.733438 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:26.733365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:52.730782 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:52.730749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:58.462128 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:58.462076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:58.524778 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:58.523803 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:58.617499 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:58.617471 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:13:58.732903 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:13:58.732824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:14.623657 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:14.623621 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:20.491494 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:20.491404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:20.555489 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:20.555461 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:20.633808 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:20.633775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:20.727132 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:20.727103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:31.221260 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:31.221207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:32.510102 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:32.510067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:37.725685 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:37.725652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:37.786932 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:37.786909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:37.869042 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:37.869020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:37.968170 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:37.968148 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:38.068417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:38.068391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:38.812872 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:38.812838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:38.912159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:38.912129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:39.033952 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:39.033929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:40.693516 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:40.693482 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:46.268413 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:46.268379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:46.339059 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:46.339030 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:46.955445 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:46.955410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:52.790586 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:52.790554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:52.911090 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:52.911057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:53.041289 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.041201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:53.316997 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.316925 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:14:53.317320 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.317306 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" containerName="istio-proxy" Jul 16 00:14:53.317372 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.317323 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" containerName="istio-proxy" Jul 16 00:14:53.317407 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.317385 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89803e4-e640-4128-8b7c-3d5c5fdacb58" containerName="istio-proxy" Jul 16 00:14:53.320429 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.320413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.323077 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.323056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9ed299d2-openshift-default-dockercfg-dncxn\"" Jul 16 00:14:53.329004 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.328977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:53.340289 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.340266 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:14:53.362151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362314 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29dcl\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362421 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362421 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362421 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362421 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.362544 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.362449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463361 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463361 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29dcl\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463940 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463940 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463807 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463940 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463862 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.463940 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.463928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.464196 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.464175 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.464320 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.464302 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.465959 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.465933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.466083 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.465965 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.472543 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.472500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:53.473044 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.473010 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29dcl\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.473297 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.473267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token\") pod \"e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.630678 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.630586 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:53.761433 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.761405 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:14:53.763931 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:14:53.763900 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6df5de6_419a_4968_acc8_75b4bd2a846d.slice/crio-621fc376e2225ff2bcf705805435adffcfd11fc6b3cd9d5067dae3e33397288f WatchSource:0}: Error finding container 621fc376e2225ff2bcf705805435adffcfd11fc6b3cd9d5067dae3e33397288f: Status 404 returned error can't find the container with id 621fc376e2225ff2bcf705805435adffcfd11fc6b3cd9d5067dae3e33397288f Jul 16 00:14:53.765952 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.765912 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:14:53.766019 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.766004 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:14:53.766080 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:53.766063 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:14:54.671815 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:54.671766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerStarted","Data":"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f"} Jul 16 00:14:54.672215 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:54.671832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerStarted","Data":"621fc376e2225ff2bcf705805435adffcfd11fc6b3cd9d5067dae3e33397288f"} Jul 16 00:14:54.693829 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:54.693778 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podStartSLOduration=1.693758881 podStartE2EDuration="1.693758881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:14:54.692199109 +0000 UTC m=+1649.747529194" watchObservedRunningTime="2026-07-16 00:14:54.693758881 +0000 UTC m=+1649.749089028" Jul 16 00:14:55.614914 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.614880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:55.631552 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.631519 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:55.636544 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.636515 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:55.676178 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.676151 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:55.677303 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.677286 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:14:55.698843 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.698801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:55.782826 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.782795 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:55.960341 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:55.960254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:56.249614 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:56.249583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:14:56.355052 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:14:56.355016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:01.108481 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:01.108436 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:01.807868 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:01.807831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:01.907702 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:01.907662 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:02.315246 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:02.315183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:08.095970 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:08.095866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:08.157901 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:08.157868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:08.788339 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:08.788304 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:10.678018 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.677985 2580 patch_prober.go:28] interesting pod/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 00:15:10.678385 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.678047 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 00:15:10.744186 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.744145 2580 generic.go:356] "Generic (PLEG): container finished" podID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerID="b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f" exitCode=0 Jul 16 00:15:10.744327 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.744207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerDied","Data":"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f"} Jul 16 00:15:10.744611 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.744597 2580 scope.go:117] "RemoveContainer" containerID="b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f" Jul 16 00:15:10.745590 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.745567 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:15:10.745654 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.745630 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:15:10.745696 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:10.745668 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:15:11.754966 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:11.753674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerStarted","Data":"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef"} Jul 16 00:15:12.632018 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:12.631973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:15:12.632263 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:12.632136 2580 patch_prober.go:28] interesting pod/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 00:15:12.632263 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:12.632183 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 00:15:13.631266 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:13.631202 2580 patch_prober.go:28] interesting pod/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 00:15:13.631743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:13.631293 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 00:15:14.599589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.599555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:14.631982 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.631949 2580 patch_prober.go:28] interesting pod/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 00:15:14.632546 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.632516 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 00:15:14.700946 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.700915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:14.898172 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.898085 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:15:14.898324 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.898300 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" containerID="cri-o://e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef" gracePeriod=30 Jul 16 00:15:14.972138 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:14.972103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:15.110576 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:15.110545 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:15.227721 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:15.227637 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:15.326606 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:15.326575 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:15.472065 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:15.472026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:21.045333 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.045311 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:15:21.053924 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.053901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:21.133159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133105 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133135 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133156 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133196 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29dcl\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133246 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133268 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133298 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133570 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133431 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133570 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133475 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs\") pod \"e6df5de6-419a-4968-acc8-75b4bd2a846d\" (UID: \"e6df5de6-419a-4968-acc8-75b4bd2a846d\") " Jul 16 00:15:21.133570 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133516 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:15:21.133736 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133707 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:15:21.133830 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133810 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.133887 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133837 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.133887 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.133875 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:15:21.134085 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.134056 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:15:21.134468 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.134441 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:15:21.135396 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.135370 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:15:21.135472 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.135411 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:15:21.135523 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.135474 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:15:21.135680 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.135664 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl" (OuterVolumeSpecName: "kube-api-access-29dcl") pod "e6df5de6-419a-4968-acc8-75b4bd2a846d" (UID: "e6df5de6-419a-4968-acc8-75b4bd2a846d"). InnerVolumeSpecName "kube-api-access-29dcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:15:21.161424 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.161404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:21.234705 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234684 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234705 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234703 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234822 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234713 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29dcl\" (UniqueName: \"kubernetes.io/projected/e6df5de6-419a-4968-acc8-75b4bd2a846d-kube-api-access-29dcl\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234822 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234724 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6df5de6-419a-4968-acc8-75b4bd2a846d-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234822 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234733 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234822 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234740 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.234822 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.234748 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6df5de6-419a-4968-acc8-75b4bd2a846d-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:15:21.255090 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.255070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:21.414878 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.414823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:21.803345 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.803320 2580 generic.go:356] "Generic (PLEG): container finished" podID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerID="e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef" exitCode=0 Jul 16 00:15:21.803473 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.803369 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" Jul 16 00:15:21.803473 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.803399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerDied","Data":"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef"} Jul 16 00:15:21.803473 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.803455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx" event={"ID":"e6df5de6-419a-4968-acc8-75b4bd2a846d","Type":"ContainerDied","Data":"621fc376e2225ff2bcf705805435adffcfd11fc6b3cd9d5067dae3e33397288f"} Jul 16 00:15:21.803576 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.803478 2580 scope.go:117] "RemoveContainer" containerID="e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef" Jul 16 00:15:21.812182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.812163 2580 scope.go:117] "RemoveContainer" containerID="b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f" Jul 16 00:15:21.819642 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.819607 2580 scope.go:117] "RemoveContainer" containerID="e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef" Jul 16 00:15:21.819897 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:15:21.819878 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef\": container with ID starting with e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef not found: ID does not exist" containerID="e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef" Jul 16 00:15:21.819944 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.819906 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef"} err="failed to get container status \"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef\": rpc error: code = NotFound desc = could not find container \"e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef\": container with ID starting with e412b512c86307dd2089a3b87b32d16aca0c09d17ad9dc623105ef9bdfcfb3ef not found: ID does not exist" Jul 16 00:15:21.819944 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.819922 2580 scope.go:117] "RemoveContainer" containerID="b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f" Jul 16 00:15:21.820210 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:15:21.820182 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f\": container with ID starting with b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f not found: ID does not exist" containerID="b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f" Jul 16 00:15:21.820273 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.820207 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f"} err="failed to get container status \"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f\": rpc error: code = NotFound desc = could not find container \"b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f\": container with ID starting with b0b610164f333f5e61a301f97c072506f455aba3c90a1a75b4c7b84390f6230f not found: ID does not exist" Jul 16 00:15:21.828418 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.828397 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:15:21.834478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:21.834458 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9ed299d2-openshift-default-7d7cc985d7-7mfmx"] Jul 16 00:15:23.509226 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:23.509178 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" path="/var/lib/kubelet/pods/e6df5de6-419a-4968-acc8-75b4bd2a846d/volumes" Jul 16 00:15:27.036571 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:27.036541 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:27.134505 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:27.134473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:34.818297 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:34.818262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:34.920970 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:34.920937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:35.016616 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:35.016592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:35.762765 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:35.762731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:35.857712 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:35.857681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:35.957086 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:35.957055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:37.363988 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:37.363954 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:42.837118 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:42.837085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:42.940068 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:42.940044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:43.352015 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:43.351981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:43.458700 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:43.458672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:44.059614 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:44.059586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:49.747719 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:49.747632 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:49.853052 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:49.853023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:15:49.948391 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:15:49.948362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:00.438151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:00.438121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:00.786951 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:00.786920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:21.764044 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:21.764006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:27.678069 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.678031 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:27.779544 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.779517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:27.916446 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916414 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:16:27.916750 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916739 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.916797 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916752 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.916797 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916765 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.916797 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916771 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.916894 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916830 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.916894 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.916838 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6df5de6-419a-4968-acc8-75b4bd2a846d" containerName="istio-proxy" Jul 16 00:16:27.918620 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.918604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:27.921234 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.921200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-30d4db-openshift-default-dockercfg-8df2k\"" Jul 16 00:16:27.931342 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.931293 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:16:27.941752 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:27.941700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:28.080995 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.080962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081156 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081156 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081265 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081265 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tq94\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081378 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.081516 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.081412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182019 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.181939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182019 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.181970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182019 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tq94\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182336 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182433 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182740 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182740 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.182828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.182812 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.184589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.184566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.184715 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.184616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.193590 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.193567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tq94\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.193712 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.193610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token\") pod \"e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.228569 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.228545 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:28.358242 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.358202 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:16:28.360954 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:16:28.360929 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966b0759_3379_493f_82ce_e1acd81e8196.slice/crio-3a9955f2ff5e9eedae1f81414a7f8cd94d012345967a282c2cf5972b526d8bb0 WatchSource:0}: Error finding container 3a9955f2ff5e9eedae1f81414a7f8cd94d012345967a282c2cf5972b526d8bb0: Status 404 returned error can't find the container with id 3a9955f2ff5e9eedae1f81414a7f8cd94d012345967a282c2cf5972b526d8bb0 Jul 16 00:16:28.363105 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.363067 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:28.363184 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.363127 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:28.363184 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.363154 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:28.645845 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:28.645804 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:29.091680 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.091651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerStarted","Data":"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6"} Jul 16 00:16:29.092074 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.091710 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerStarted","Data":"3a9955f2ff5e9eedae1f81414a7f8cd94d012345967a282c2cf5972b526d8bb0"} Jul 16 00:16:29.110547 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.110504 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podStartSLOduration=2.110488874 podStartE2EDuration="2.110488874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:16:29.108479387 +0000 UTC m=+1744.163809485" watchObservedRunningTime="2026-07-16 00:16:29.110488874 +0000 UTC m=+1744.165819012" Jul 16 00:16:29.229576 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.229549 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:29.230885 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.230862 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:29.230993 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:29.230910 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:30.056627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.056598 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:30.144726 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.144699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:30.214869 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.214811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:30.232813 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.232791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:30.432711 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.432623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:30.765371 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.765340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:30.859735 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:30.859706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:31.101177 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:31.101091 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:31.101905 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:31.101888 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:35.578944 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:35.578910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:35.926991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:35.926918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:36.228072 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:36.227994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:36.360070 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:36.360042 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:37.213085 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:37.213050 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:37.780714 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:37.780668 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:37.870343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:37.870305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:37.960075 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:37.960049 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:38.059656 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:38.059463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:38.536490 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:38.536460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:38.636927 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:38.636902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:43.300118 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:43.300084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:47.142389 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:47.142351 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 00:16:47.142773 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:47.142456 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 00:16:49.186588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.186555 2580 generic.go:356] "Generic (PLEG): container finished" podID="966b0759-3379-493f-82ce-e1acd81e8196" containerID="f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6" exitCode=0 Jul 16 00:16:49.186992 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.186625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerDied","Data":"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6"} Jul 16 00:16:49.187040 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.187008 2580 scope.go:117] "RemoveContainer" containerID="f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6" Jul 16 00:16:49.188056 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.188009 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:49.188159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.188089 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:49.188159 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:49.188133 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:16:50.195495 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:50.195466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerStarted","Data":"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6"} Jul 16 00:16:50.228880 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:50.228852 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:50.229092 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:50.229071 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:50.229179 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:50.229119 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:51.229643 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:51.229604 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:51.230088 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:51.229662 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:51.498861 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:51.498821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:51.599918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:51.599879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:52.229811 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:52.229775 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:52.230300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:52.229830 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:53.228938 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:53.228899 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:53.229118 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:53.228965 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:54.229194 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:54.229162 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 00:16:54.229668 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:54.229252 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 00:16:55.232811 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:55.232785 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:55.233287 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:55.233167 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:55.233790 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:55.233774 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:16:59.285502 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:59.285465 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:16:59.386007 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:16:59.385973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:02.228885 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:02.228857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:19.105324 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.105232 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:17:19.116574 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.116539 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.117132 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.117104 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:17:19.120588 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.120560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4dc784\"/\"test-model-4dc784-kserve-self-signed-certs\"" Jul 16 00:17:19.120792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.120758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4dc784\"/\"default-dockercfg-qg9ft\"" Jul 16 00:17:19.120907 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.120887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4dc784\"/\"kube-root-ca.crt\"" Jul 16 00:17:19.120973 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.120633 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4dc784\"/\"openshift-service-ca.crt\"" Jul 16 00:17:19.138584 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.138679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.138679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xtw\" (UniqueName: \"kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.138679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.138828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.138828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.138742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.212108 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.212079 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:19.239500 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239592 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239592 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239592 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xtw\" (UniqueName: \"kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239703 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239703 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239904 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239884 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239946 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239907 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.239992 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.239967 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.241691 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.241676 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.242047 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.242031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.247798 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.247778 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xtw\" (UniqueName: \"kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw\") pod \"test-model-4dc784-kserve-6f6655bc45-cq2k8\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.427546 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.427462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:19.553002 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.552879 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:17:19.555201 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:17:19.555177 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f1be10e_2645_49ec_957d_d9cc63a92f82.slice/crio-f950cd87cbdfeed8d6b5a57ce6ba26e939991ae0983ff53442c4fbba9bd4bcd4 WatchSource:0}: Error finding container f950cd87cbdfeed8d6b5a57ce6ba26e939991ae0983ff53442c4fbba9bd4bcd4: Status 404 returned error can't find the container with id f950cd87cbdfeed8d6b5a57ce6ba26e939991ae0983ff53442c4fbba9bd4bcd4 Jul 16 00:17:19.557132 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:19.557116 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:17:20.329932 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:20.329902 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" event={"ID":"0f1be10e-2645-49ec-957d-d9cc63a92f82","Type":"ContainerStarted","Data":"f950cd87cbdfeed8d6b5a57ce6ba26e939991ae0983ff53442c4fbba9bd4bcd4"} Jul 16 00:17:21.336814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:21.336783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" event={"ID":"0f1be10e-2645-49ec-957d-d9cc63a92f82","Type":"ContainerStarted","Data":"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4"} Jul 16 00:17:21.354187 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:21.354125 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" podStartSLOduration=1.255065817 podStartE2EDuration="2.354108415s" totalImagesPullingTime="1.099042598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:21.35385939 +0000 UTC m=+1796.409189480" watchObservedRunningTime="2026-07-16 00:17:21.354108415 +0000 UTC m=+1796.409438489" Jul 16 00:17:29.428205 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:29.428168 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:29.428205 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:29.428213 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:29.440089 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:29.440059 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:30.386856 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:30.386828 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:17:52.141417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:52.141386 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:53.083556 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:53.083518 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:17:53.097539 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:53.097514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:53.637914 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:53.637885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:17:58.868828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:17:58.868798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:08.102792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.102735 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" podUID="0f1be10e-2645-49ec-957d-d9cc63a92f82" containerName="main" containerID="cri-o://62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4" gracePeriod=45 Jul 16 00:18:08.335101 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.335081 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:18:08.417291 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417236 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7xtw\" (UniqueName: \"kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417291 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417280 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417456 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417313 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417456 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417329 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417456 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417372 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417456 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417404 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir\") pod \"0f1be10e-2645-49ec-957d-d9cc63a92f82\" (UID: \"0f1be10e-2645-49ec-957d-d9cc63a92f82\") " Jul 16 00:18:08.417651 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417596 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home" (OuterVolumeSpecName: "home") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:08.417714 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417666 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache" (OuterVolumeSpecName: "model-cache") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:08.417778 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.417751 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:08.419471 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.419453 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm" (OuterVolumeSpecName: "dshm") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:08.419471 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.419462 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw" (OuterVolumeSpecName: "kube-api-access-h7xtw") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "kube-api-access-h7xtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:18:08.419595 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.419485 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f1be10e-2645-49ec-957d-d9cc63a92f82" (UID: "0f1be10e-2645-49ec-957d-d9cc63a92f82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:18:08.518089 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518069 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.518089 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518086 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7xtw\" (UniqueName: \"kubernetes.io/projected/0f1be10e-2645-49ec-957d-d9cc63a92f82-kube-api-access-h7xtw\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.518200 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518095 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1be10e-2645-49ec-957d-d9cc63a92f82-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.518200 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518103 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.518200 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518110 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.518200 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.518117 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1be10e-2645-49ec-957d-d9cc63a92f82-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:08.533175 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.533151 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f1be10e-2645-49ec-957d-d9cc63a92f82" containerID="62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4" exitCode=0 Jul 16 00:18:08.533264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.533187 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" event={"ID":"0f1be10e-2645-49ec-957d-d9cc63a92f82","Type":"ContainerDied","Data":"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4"} Jul 16 00:18:08.533264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.533195 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" Jul 16 00:18:08.533264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.533229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8" event={"ID":"0f1be10e-2645-49ec-957d-d9cc63a92f82","Type":"ContainerDied","Data":"f950cd87cbdfeed8d6b5a57ce6ba26e939991ae0983ff53442c4fbba9bd4bcd4"} Jul 16 00:18:08.533264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.533253 2580 scope.go:117] "RemoveContainer" containerID="62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4" Jul 16 00:18:08.541349 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.541333 2580 scope.go:117] "RemoveContainer" containerID="62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4" Jul 16 00:18:08.541571 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:18:08.541551 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4\": container with ID starting with 62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4 not found: ID does not exist" containerID="62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4" Jul 16 00:18:08.541611 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.541579 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4"} err="failed to get container status \"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4\": rpc error: code = NotFound desc = could not find container \"62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4\": container with ID starting with 62bf4f4529f606419b80ca5f70f3acd3b4ec5cf4ebadab87c70c1a88286e61d4 not found: ID does not exist" Jul 16 00:18:08.556933 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.556908 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:18:08.562885 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:08.562867 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4dc784/test-model-4dc784-kserve-6f6655bc45-cq2k8"] Jul 16 00:18:09.508647 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:09.508618 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f1be10e-2645-49ec-957d-d9cc63a92f82" path="/var/lib/kubelet/pods/0f1be10e-2645-49ec-957d-d9cc63a92f82/volumes" Jul 16 00:18:36.037946 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:36.037905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:36.347468 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:36.347395 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:18:36.347609 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:36.347588 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" containerID="cri-o://3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6" gracePeriod=30 Jul 16 00:18:36.425307 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:36.425271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:36.633146 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:36.633074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:37.089848 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:37.089823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:40.235589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:40.235555 2580 patch_prober.go:28] interesting pod/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 00:18:40.235925 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:40.235629 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:18:41.591863 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.591839 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:18:41.660093 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660038 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660198 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660097 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660198 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660112 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660212 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660318 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660517 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660372 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660517 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660422 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tq94\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660517 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660460 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert\") pod \"966b0759-3379-493f-82ce-e1acd81e8196\" (UID: \"966b0759-3379-493f-82ce-e1acd81e8196\") " Jul 16 00:18:41.660791 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660456 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:41.660791 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660782 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:18:41.660909 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660491 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:41.660909 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.660848 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:41.661068 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.661048 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data" (OuterVolumeSpecName: "istio-data") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:41.662279 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.662231 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token" (OuterVolumeSpecName: "istio-token") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:18:41.662279 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.662234 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:18:41.662704 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.662686 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:18:41.662754 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.662689 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94" (OuterVolumeSpecName: "kube-api-access-9tq94") pod "966b0759-3379-493f-82ce-e1acd81e8196" (UID: "966b0759-3379-493f-82ce-e1acd81e8196"). InnerVolumeSpecName "kube-api-access-9tq94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:18:41.677333 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.677309 2580 generic.go:356] "Generic (PLEG): container finished" podID="966b0759-3379-493f-82ce-e1acd81e8196" containerID="3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6" exitCode=0 Jul 16 00:18:41.677417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.677351 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" Jul 16 00:18:41.677417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.677369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerDied","Data":"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6"} Jul 16 00:18:41.677417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.677409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f" event={"ID":"966b0759-3379-493f-82ce-e1acd81e8196","Type":"ContainerDied","Data":"3a9955f2ff5e9eedae1f81414a7f8cd94d012345967a282c2cf5972b526d8bb0"} Jul 16 00:18:41.677526 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.677430 2580 scope.go:117] "RemoveContainer" containerID="3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6" Jul 16 00:18:41.685601 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.685584 2580 scope.go:117] "RemoveContainer" containerID="f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6" Jul 16 00:18:41.692800 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.692784 2580 scope.go:117] "RemoveContainer" containerID="3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6" Jul 16 00:18:41.693061 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:18:41.693041 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6\": container with ID starting with 3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6 not found: ID does not exist" containerID="3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6" Jul 16 00:18:41.693107 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.693068 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6"} err="failed to get container status \"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6\": rpc error: code = NotFound desc = could not find container \"3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6\": container with ID starting with 3f142c30f0ea7d69f49e3cfe17fdc192428e95713292402ad933e4023e662ad6 not found: ID does not exist" Jul 16 00:18:41.693107 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.693087 2580 scope.go:117] "RemoveContainer" containerID="f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6" Jul 16 00:18:41.693398 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:18:41.693383 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6\": container with ID starting with f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6 not found: ID does not exist" containerID="f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6" Jul 16 00:18:41.693443 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.693404 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6"} err="failed to get container status \"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6\": rpc error: code = NotFound desc = could not find container \"f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6\": container with ID starting with f71696220738546e9b79b8e6be0fc22aac7714b094d68b243bed75de5200f7b6 not found: ID does not exist" Jul 16 00:18:41.703061 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.703042 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:18:41.707427 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.707409 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-30d4db-openshift-default-5654875975-dzh4f"] Jul 16 00:18:41.761725 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761704 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761725 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761725 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761734 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761742 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966b0759-3379-493f-82ce-e1acd81e8196-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761750 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761757 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761765 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966b0759-3379-493f-82ce-e1acd81e8196-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761772 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tq94\" (UniqueName: \"kubernetes.io/projected/966b0759-3379-493f-82ce-e1acd81e8196-kube-api-access-9tq94\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:41.761844 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:41.761780 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966b0759-3379-493f-82ce-e1acd81e8196-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:18:43.391972 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:43.391942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:43.502869 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:43.502837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:43.511425 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:43.509156 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966b0759-3379-493f-82ce-e1acd81e8196" path="/var/lib/kubelet/pods/966b0759-3379-493f-82ce-e1acd81e8196/volumes" Jul 16 00:18:43.676494 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:43.676430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:43.891495 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:43.891443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:44.261248 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.261203 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:44.568344 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568276 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568638 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568651 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568659 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1be10e-2645-49ec-957d-d9cc63a92f82" containerName="main" Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568665 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1be10e-2645-49ec-957d-d9cc63a92f82" containerName="main" Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568674 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568679 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568865 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568726 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568865 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568735 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="966b0759-3379-493f-82ce-e1acd81e8196" containerName="istio-proxy" Jul 16 00:18:44.568865 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.568741 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1be10e-2645-49ec-957d-d9cc63a92f82" containerName="main" Jul 16 00:18:44.573381 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.573364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.579807 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.579596 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2aa494-openshift-default-dockercfg-znjw4\"" Jul 16 00:18:44.586068 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.586037 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:18:44.592201 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.592156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:44.683021 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.682993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683164 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683164 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kkdp\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683164 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683300 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.683419 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.683312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784175 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784334 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784642 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kkdp\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784642 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784642 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784779 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784665 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784835 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784819 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784891 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784840 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.784979 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.784961 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.786529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.786505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.786738 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.786722 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.793053 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.793032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.793503 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.793484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kkdp\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp\") pod \"e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:44.806235 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.806199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:44.884414 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:44.884364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:45.013914 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.013879 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:18:45.016837 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:18:45.016809 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ea8a9d_abbb_4c46_bb5e_8221df169863.slice/crio-6cb658d8e6f2dbadd94fc588b9781f9b3070dbec422b1384b64235180cc560bb WatchSource:0}: Error finding container 6cb658d8e6f2dbadd94fc588b9781f9b3070dbec422b1384b64235180cc560bb: Status 404 returned error can't find the container with id 6cb658d8e6f2dbadd94fc588b9781f9b3070dbec422b1384b64235180cc560bb Jul 16 00:18:45.018849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.018816 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:18:45.018942 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.018882 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:18:45.018942 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.018910 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:18:45.694271 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.694241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerStarted","Data":"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5"} Jul 16 00:18:45.694610 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.694284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerStarted","Data":"6cb658d8e6f2dbadd94fc588b9781f9b3070dbec422b1384b64235180cc560bb"} Jul 16 00:18:45.734105 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.734063 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podStartSLOduration=1.7340480889999998 podStartE2EDuration="1.734048089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:18:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:18:45.733319957 +0000 UTC m=+1880.788650040" watchObservedRunningTime="2026-07-16 00:18:45.734048089 +0000 UTC m=+1880.789378160" Jul 16 00:18:45.885261 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:45.885235 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:46.754807 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:46.754771 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:46.860450 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:46.860424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:46.889956 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:46.889934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:46.986761 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:46.986732 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:47.219762 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:47.219652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:47.649520 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:47.649488 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:47.702394 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:47.702369 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:47.703280 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:47.703261 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:18:47.779180 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:47.779154 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:52.248442 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:52.248409 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:52.806013 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:52.805971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:53.140634 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:53.140548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:53.439241 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:53.439142 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:54.667587 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:54.667555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:54.791326 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:54.791300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:54.882705 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:54.882671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:55.161133 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:55.161103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:55.427060 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:55.426946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:55.549560 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:55.549527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:18:59.761203 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.761131 2580 generic.go:356] "Generic (PLEG): container finished" podID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerID="cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5" exitCode=0 Jul 16 00:18:59.761203 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.761183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerDied","Data":"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5"} Jul 16 00:18:59.761638 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.761545 2580 scope.go:117] "RemoveContainer" containerID="cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5" Jul 16 00:18:59.762496 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.762469 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:18:59.762563 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.762532 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:18:59.762604 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:18:59.762562 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:00.200185 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:00.200109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:00.767607 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:00.767578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerStarted","Data":"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47"} Jul 16 00:19:00.885213 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:00.885182 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:00.886250 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:00.886210 2580 patch_prober.go:28] interesting pod/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 00:19:00.886349 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:00.886285 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 00:19:01.885705 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:01.885666 2580 patch_prober.go:28] interesting pod/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 00:19:01.886181 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:01.885762 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 00:19:02.702792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:02.702752 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:02.884889 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:02.884856 2580 patch_prober.go:28] interesting pod/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 00:19:02.885052 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:02.884905 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 00:19:03.885175 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:03.885141 2580 patch_prober.go:28] interesting pod/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 00:19:03.885567 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:03.885193 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 00:19:04.889040 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:04.889003 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:04.890020 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:04.889999 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:08.679578 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:08.679546 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:08.780343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:08.780309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:08.891190 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:08.891158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:14.872680 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:14.872645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:16.339968 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:16.339940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:16.442049 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:16.442019 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:20.581069 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:20.581034 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:20.654501 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:20.654473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:20.850991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:20.850913 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:19:20.851169 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:20.851143 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" containerID="cri-o://9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47" gracePeriod=30 Jul 16 00:19:20.880577 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:20.880545 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:21.093494 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:21.093462 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:21.586156 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:21.586130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:24.950123 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:24.950089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:25.522019 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:25.521989 2580 scope.go:117] "RemoveContainer" containerID="8b9a0c89ebb30c06d050d3da4a24bba884f5b4fbe0ebe28aa13805eac228266a" Jul 16 00:19:26.098629 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.098603 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:26.155803 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155747 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155803 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155783 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155810 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155829 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155889 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155917 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kkdp\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155943 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.155991 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.155968 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.156307 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156003 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token\") pod \"05ea8a9d-abbb-4c46-bb5e-8221df169863\" (UID: \"05ea8a9d-abbb-4c46-bb5e-8221df169863\") " Jul 16 00:19:26.156307 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156099 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:26.156307 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156110 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:26.156307 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156145 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:26.156507 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156401 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.156507 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156421 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ea8a9d-abbb-4c46-bb5e-8221df169863-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.156507 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156435 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.156507 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156490 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:26.156810 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.156791 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data" (OuterVolumeSpecName: "istio-data") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:26.158154 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.158133 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:19:26.158154 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.158141 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp" (OuterVolumeSpecName: "kube-api-access-4kkdp") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "kube-api-access-4kkdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:19:26.158435 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.158415 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:26.158495 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.158454 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token" (OuterVolumeSpecName: "istio-token") pod "05ea8a9d-abbb-4c46-bb5e-8221df169863" (UID: "05ea8a9d-abbb-4c46-bb5e-8221df169863"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:19:26.256918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256884 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.256918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256906 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.256918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256916 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kkdp\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-kube-api-access-4kkdp\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.256918 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256925 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.257172 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256934 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.257172 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.256941 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ea8a9d-abbb-4c46-bb5e-8221df169863-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:19:26.883290 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.883257 2580 generic.go:356] "Generic (PLEG): container finished" podID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerID="9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47" exitCode=0 Jul 16 00:19:26.883478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.883303 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" Jul 16 00:19:26.883478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.883342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerDied","Data":"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47"} Jul 16 00:19:26.883478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.883407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v" event={"ID":"05ea8a9d-abbb-4c46-bb5e-8221df169863","Type":"ContainerDied","Data":"6cb658d8e6f2dbadd94fc588b9781f9b3070dbec422b1384b64235180cc560bb"} Jul 16 00:19:26.883478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.883438 2580 scope.go:117] "RemoveContainer" containerID="9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47" Jul 16 00:19:26.901107 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.901080 2580 scope.go:117] "RemoveContainer" containerID="cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5" Jul 16 00:19:26.908995 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.908978 2580 scope.go:117] "RemoveContainer" containerID="9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47" Jul 16 00:19:26.909478 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:19:26.909446 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47\": container with ID starting with 9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47 not found: ID does not exist" containerID="9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47" Jul 16 00:19:26.909602 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.909480 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47"} err="failed to get container status \"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47\": rpc error: code = NotFound desc = could not find container \"9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47\": container with ID starting with 9810152f05d09d50ad76529d9c7e1b33ae7ebb0356d3644069aa70a286759d47 not found: ID does not exist" Jul 16 00:19:26.909602 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.909505 2580 scope.go:117] "RemoveContainer" containerID="cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5" Jul 16 00:19:26.909907 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:19:26.909874 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5\": container with ID starting with cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5 not found: ID does not exist" containerID="cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5" Jul 16 00:19:26.910033 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.909915 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5"} err="failed to get container status \"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5\": rpc error: code = NotFound desc = could not find container \"cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5\": container with ID starting with cceaae0f8308427b69976c5852308265570807f17d529e60dc7d0f03c5ffe7a5 not found: ID does not exist" Jul 16 00:19:26.911350 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.911331 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:19:26.919717 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:26.919696 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2aa494-openshift-default-59bbb9c4f-25x2v"] Jul 16 00:19:27.509065 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:27.509035 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" path="/var/lib/kubelet/pods/05ea8a9d-abbb-4c46-bb5e-8221df169863/volumes" Jul 16 00:19:27.577799 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:27.577774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:33.346684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:33.346648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:34.701517 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.701477 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:19:34.701886 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.701804 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:19:34.701886 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.701815 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:19:34.701886 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.701878 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:19:34.701886 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.701886 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:19:34.704771 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.704754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.707631 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.707610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-26b6c6-openshift-default-dockercfg-74pps\"" Jul 16 00:19:34.718889 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.718866 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:19:34.821917 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.821883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.821917 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.821919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822150 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.821940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822150 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822150 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822150 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822085 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822150 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpsrz\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822373 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.822373 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.822245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923092 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923293 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923293 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923293 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpsrz\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923293 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923293 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923710 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923691 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923798 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923861 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.923919 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.923873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.924102 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.924067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.925548 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.925524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.926046 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.926027 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.931411 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.931390 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpsrz\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:34.931490 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:34.931460 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token\") pod \"e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:35.014578 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.014557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:35.052531 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.052497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:35.146509 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.146476 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:19:35.148696 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:19:35.148664 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a4e1b8_cf14_4dc8_9506_dcd6835c986c.slice/crio-7db462c2e35f2673413bf96f5839856756a576f40499d5a4e31cee115ef1d439 WatchSource:0}: Error finding container 7db462c2e35f2673413bf96f5839856756a576f40499d5a4e31cee115ef1d439: Status 404 returned error can't find the container with id 7db462c2e35f2673413bf96f5839856756a576f40499d5a4e31cee115ef1d439 Jul 16 00:19:35.150856 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.150821 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:35.150960 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.150888 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:35.150960 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.150917 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:35.231971 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.231944 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:35.922944 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.922904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerStarted","Data":"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63"} Jul 16 00:19:35.923344 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.922963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerStarted","Data":"7db462c2e35f2673413bf96f5839856756a576f40499d5a4e31cee115ef1d439"} Jul 16 00:19:35.948636 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:35.948586 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" podStartSLOduration=1.948567942 podStartE2EDuration="1.948567942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:19:35.948166099 +0000 UTC m=+1931.003496172" watchObservedRunningTime="2026-07-16 00:19:35.948567942 +0000 UTC m=+1931.003898015" Jul 16 00:19:36.014828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.014799 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:36.019612 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.019592 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:36.871510 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.871476 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:36.927756 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.927726 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:36.928804 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.928780 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:36.948043 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:36.948016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:37.016401 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:37.016370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:37.274186 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:37.274157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:37.535469 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:37.535396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:37.639060 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:37.639030 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:42.471206 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:42.471170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:42.864819 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:42.864782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:43.161079 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:43.161012 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:43.357376 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:43.357345 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:44.981057 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:44.981023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:45.062992 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:45.062964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:45.128172 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:45.128144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:45.363976 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:45.363948 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:45.540478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:45.540448 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:45.640385 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:45.640322 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:50.477783 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:50.477752 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:56.012568 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.012535 2580 generic.go:356] "Generic (PLEG): container finished" podID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerID="03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63" exitCode=0 Jul 16 00:19:56.013009 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.012612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerDied","Data":"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63"} Jul 16 00:19:56.013009 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.012971 2580 scope.go:117] "RemoveContainer" containerID="03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63" Jul 16 00:19:56.013970 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.013941 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:56.014090 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.014029 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:56.014090 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:56.014080 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:19:57.019027 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:57.018994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerStarted","Data":"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728"} Jul 16 00:19:58.015516 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:58.015483 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:19:58.015734 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:58.015713 2580 patch_prober.go:28] interesting pod/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:19:58.015781 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:58.015757 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 00:19:58.333414 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:58.333325 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:58.435851 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:58.435813 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:19:59.015181 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:59.015140 2580 patch_prober.go:28] interesting pod/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:19:59.015383 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:19:59.015212 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 00:20:00.015536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:00.015504 2580 patch_prober.go:28] interesting pod/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:20:00.015905 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:00.015565 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 00:20:01.018743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:01.018711 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:20:01.019115 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:01.019097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:20:01.019915 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:01.019894 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:20:06.351915 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:06.351885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:06.447838 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:06.447800 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:20.107489 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.107400 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:20:20.107829 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.107772 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:20:20.107829 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.107785 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ea8a9d-abbb-4c46-bb5e-8221df169863" containerName="istio-proxy" Jul 16 00:20:20.109807 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.109790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.112668 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.112642 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"kube-root-ca.crt\"" Jul 16 00:20:20.113048 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.113032 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"openshift-service-ca.crt\"" Jul 16 00:20:20.113980 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.113958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"auth-test-model-26b6c6-kserve-self-signed-certs\"" Jul 16 00:20:20.114089 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.114040 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"default-dockercfg-b4v4l\"" Jul 16 00:20:20.121665 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.121643 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:20:20.150668 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.150638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:20.206948 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.206919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.207120 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.206975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5fwf\" (UniqueName: \"kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.207120 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.207041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.207250 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.207124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.207250 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.207148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.207250 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.207198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308198 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5fwf\" (UniqueName: \"kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308395 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308395 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308504 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308504 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308612 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308694 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308673 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308802 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308780 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.308863 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.308817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.310606 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.310584 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.310838 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.310821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.316485 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.316460 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5fwf\" (UniqueName: \"kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf\") pod \"auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.420350 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.420292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:20.543041 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:20.543014 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:20:20.545814 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:20:20.545790 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62fd5a1e_f180_485d_b531_594bd6081af8.slice/crio-346ed56da2ff160148306a3adc5f3706993f21b431f86194192d130e5819f715 WatchSource:0}: Error finding container 346ed56da2ff160148306a3adc5f3706993f21b431f86194192d130e5819f715: Status 404 returned error can't find the container with id 346ed56da2ff160148306a3adc5f3706993f21b431f86194192d130e5819f715 Jul 16 00:20:21.124750 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:21.124714 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" event={"ID":"62fd5a1e-f180-485d-b531-594bd6081af8","Type":"ContainerStarted","Data":"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c"} Jul 16 00:20:21.124750 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:21.124756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" event={"ID":"62fd5a1e-f180-485d-b531-594bd6081af8","Type":"ContainerStarted","Data":"346ed56da2ff160148306a3adc5f3706993f21b431f86194192d130e5819f715"} Jul 16 00:20:21.142229 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:21.142179 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" podStartSLOduration=1.142161511 podStartE2EDuration="1.142161511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:20:21.141526217 +0000 UTC m=+1976.196856291" watchObservedRunningTime="2026-07-16 00:20:21.142161511 +0000 UTC m=+1976.197491603" Jul 16 00:20:30.421375 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:30.421343 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:30.421375 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:30.421381 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:30.433743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:30.433717 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:31.177421 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:31.177390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:20:38.012785 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.012752 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:20:38.014903 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.014881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.018828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.018804 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"kube-root-ca.crt\"" Jul 16 00:20:38.018828 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.018817 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"default-dockercfg-2hxht\"" Jul 16 00:20:38.019014 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.018833 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"auth-test-model-c688a9-kserve-self-signed-certs\"" Jul 16 00:20:38.019014 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.018849 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"openshift-service-ca.crt\"" Jul 16 00:20:38.027739 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.027713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:20:38.034842 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.034816 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:38.158928 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.158899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.159066 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.158942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.159066 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.158965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.159066 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.159028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwxg4\" (UniqueName: \"kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.159169 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.159113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.159169 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.159150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260448 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwxg4\" (UniqueName: \"kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260627 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.260937 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.261003 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.261003 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.260986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.262877 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.262830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.263252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.263213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.268725 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.268699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwxg4\" (UniqueName: \"kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4\") pod \"auth-test-model-c688a9-kserve-864c9d9455-rd8qb\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.327887 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.327867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:38.453538 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:38.453514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:20:38.456206 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:20:38.456179 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod541e1318_864e_4e89_b75c_ff9e8632da09.slice/crio-756d8be7df5a114a82fba640fdde8776b05921f529b79f146c30412f00b6c437 WatchSource:0}: Error finding container 756d8be7df5a114a82fba640fdde8776b05921f529b79f146c30412f00b6c437: Status 404 returned error can't find the container with id 756d8be7df5a114a82fba640fdde8776b05921f529b79f146c30412f00b6c437 Jul 16 00:20:39.201635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:39.201600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" event={"ID":"541e1318-864e-4e89-b75c-ff9e8632da09","Type":"ContainerStarted","Data":"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb"} Jul 16 00:20:39.202033 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:39.201647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" event={"ID":"541e1318-864e-4e89-b75c-ff9e8632da09","Type":"ContainerStarted","Data":"756d8be7df5a114a82fba640fdde8776b05921f529b79f146c30412f00b6c437"} Jul 16 00:20:39.219105 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:39.219048 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" podStartSLOduration=2.219033936 podStartE2EDuration="2.219033936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:20:39.218176907 +0000 UTC m=+1994.273507005" watchObservedRunningTime="2026-07-16 00:20:39.219033936 +0000 UTC m=+1994.274364009" Jul 16 00:20:43.548426 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:43.548391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:43.653774 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:43.653747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:20:48.328501 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:48.328469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:48.328849 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:48.328516 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:48.340607 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:48.340589 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:20:49.258675 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:20:49.258648 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:21:05.135286 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:21:05.135255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:21:05.236030 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:21:05.236006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:21:59.290161 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:21:59.290125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:22:14.459633 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:22:14.459598 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:22:14.510339 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:22:14.510307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:23:00.582659 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:00.582630 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:23:00.609605 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:00.609580 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:23:01.543670 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:01.543639 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:23:01.573725 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:01.573698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:23:15.602323 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.602282 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" podUID="62fd5a1e-f180-485d-b531-594bd6081af8" containerName="main" containerID="cri-o://34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c" gracePeriod=45 Jul 16 00:23:15.842712 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.842690 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:23:15.903416 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.903355 2580 generic.go:356] "Generic (PLEG): container finished" podID="62fd5a1e-f180-485d-b531-594bd6081af8" containerID="34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c" exitCode=0 Jul 16 00:23:15.903416 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.903390 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" Jul 16 00:23:15.903416 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.903407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" event={"ID":"62fd5a1e-f180-485d-b531-594bd6081af8","Type":"ContainerDied","Data":"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c"} Jul 16 00:23:15.903582 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.903436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5" event={"ID":"62fd5a1e-f180-485d-b531-594bd6081af8","Type":"ContainerDied","Data":"346ed56da2ff160148306a3adc5f3706993f21b431f86194192d130e5819f715"} Jul 16 00:23:15.903582 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.903452 2580 scope.go:117] "RemoveContainer" containerID="34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c" Jul 16 00:23:15.911208 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.911193 2580 scope.go:117] "RemoveContainer" containerID="34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c" Jul 16 00:23:15.911466 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:23:15.911447 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c\": container with ID starting with 34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c not found: ID does not exist" containerID="34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c" Jul 16 00:23:15.911529 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.911473 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c"} err="failed to get container status \"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c\": rpc error: code = NotFound desc = could not find container \"34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c\": container with ID starting with 34d687bbc16e5a66643e77d32effa2c003686177785d010be8000709a83dc74c not found: ID does not exist" Jul 16 00:23:15.973657 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973634 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.973743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973663 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.973743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973679 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.973743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973706 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5fwf\" (UniqueName: \"kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.973910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973744 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.973910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973867 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache\") pod \"62fd5a1e-f180-485d-b531-594bd6081af8\" (UID: \"62fd5a1e-f180-485d-b531-594bd6081af8\") " Jul 16 00:23:15.974017 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973921 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:15.974017 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.973941 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home" (OuterVolumeSpecName: "home") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:15.974121 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.974094 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache" (OuterVolumeSpecName: "model-cache") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:15.974240 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.974204 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:15.974240 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.974242 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:15.974415 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.974252 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:15.975816 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.975792 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm" (OuterVolumeSpecName: "dshm") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:15.975910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.975816 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:23:15.975985 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:15.975958 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf" (OuterVolumeSpecName: "kube-api-access-z5fwf") pod "62fd5a1e-f180-485d-b531-594bd6081af8" (UID: "62fd5a1e-f180-485d-b531-594bd6081af8"). InnerVolumeSpecName "kube-api-access-z5fwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:23:16.075108 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.075083 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fd5a1e-f180-485d-b531-594bd6081af8-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.075108 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.075106 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5fwf\" (UniqueName: \"kubernetes.io/projected/62fd5a1e-f180-485d-b531-594bd6081af8-kube-api-access-z5fwf\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.075247 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.075116 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fd5a1e-f180-485d-b531-594bd6081af8-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.225472 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.225447 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:23:16.231059 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.231034 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/auth-test-model-26b6c6-kserve-6dbd57cbcf-q2mg5"] Jul 16 00:23:16.563635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.563600 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" podUID="541e1318-864e-4e89-b75c-ff9e8632da09" containerName="main" containerID="cri-o://740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb" gracePeriod=45 Jul 16 00:23:16.809187 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.809166 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:23:16.883151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883085 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883151 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883367 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883154 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883367 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883178 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwxg4\" (UniqueName: \"kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883367 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883278 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883367 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883360 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm\") pod \"541e1318-864e-4e89-b75c-ff9e8632da09\" (UID: \"541e1318-864e-4e89-b75c-ff9e8632da09\") " Jul 16 00:23:16.883561 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883388 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home" (OuterVolumeSpecName: "home") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:16.883561 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883396 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache" (OuterVolumeSpecName: "model-cache") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:16.883650 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883604 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:16.883738 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883722 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.883804 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883739 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.883804 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.883764 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.885424 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.885392 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm" (OuterVolumeSpecName: "dshm") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:23:16.885543 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.885468 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:23:16.885543 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.885472 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4" (OuterVolumeSpecName: "kube-api-access-fwxg4") pod "541e1318-864e-4e89-b75c-ff9e8632da09" (UID: "541e1318-864e-4e89-b75c-ff9e8632da09"). InnerVolumeSpecName "kube-api-access-fwxg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:23:16.909766 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.909743 2580 generic.go:356] "Generic (PLEG): container finished" podID="541e1318-864e-4e89-b75c-ff9e8632da09" containerID="740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb" exitCode=0 Jul 16 00:23:16.909866 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.909782 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" Jul 16 00:23:16.909866 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.909820 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" event={"ID":"541e1318-864e-4e89-b75c-ff9e8632da09","Type":"ContainerDied","Data":"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb"} Jul 16 00:23:16.909866 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.909858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb" event={"ID":"541e1318-864e-4e89-b75c-ff9e8632da09","Type":"ContainerDied","Data":"756d8be7df5a114a82fba640fdde8776b05921f529b79f146c30412f00b6c437"} Jul 16 00:23:16.909990 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.909875 2580 scope.go:117] "RemoveContainer" containerID="740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb" Jul 16 00:23:16.918499 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.918479 2580 scope.go:117] "RemoveContainer" containerID="740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb" Jul 16 00:23:16.918733 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:23:16.918715 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb\": container with ID starting with 740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb not found: ID does not exist" containerID="740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb" Jul 16 00:23:16.918801 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.918745 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb"} err="failed to get container status \"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb\": rpc error: code = NotFound desc = could not find container \"740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb\": container with ID starting with 740ab0cff4a62f00c2d1da8c8dc8b539f6bafd12efc397b704bfdec89956a4bb not found: ID does not exist" Jul 16 00:23:16.932923 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.932899 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:23:16.936629 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.936610 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/auth-test-model-c688a9-kserve-864c9d9455-rd8qb"] Jul 16 00:23:16.985099 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.985078 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541e1318-864e-4e89-b75c-ff9e8632da09-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.985099 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.985099 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541e1318-864e-4e89-b75c-ff9e8632da09-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:16.985255 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:16.985108 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwxg4\" (UniqueName: \"kubernetes.io/projected/541e1318-864e-4e89-b75c-ff9e8632da09-kube-api-access-fwxg4\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:23:17.510000 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:17.509968 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="541e1318-864e-4e89-b75c-ff9e8632da09" path="/var/lib/kubelet/pods/541e1318-864e-4e89-b75c-ff9e8632da09/volumes" Jul 16 00:23:17.510354 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:17.510340 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62fd5a1e-f180-485d-b531-594bd6081af8" path="/var/lib/kubelet/pods/62fd5a1e-f180-485d-b531-594bd6081af8/volumes" Jul 16 00:23:19.715183 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715156 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:23:19.715665 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715646 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62fd5a1e-f180-485d-b531-594bd6081af8" containerName="main" Jul 16 00:23:19.715746 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715669 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62fd5a1e-f180-485d-b531-594bd6081af8" containerName="main" Jul 16 00:23:19.715746 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715707 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="541e1318-864e-4e89-b75c-ff9e8632da09" containerName="main" Jul 16 00:23:19.715746 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715716 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="541e1318-864e-4e89-b75c-ff9e8632da09" containerName="main" Jul 16 00:23:19.715892 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715806 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="62fd5a1e-f180-485d-b531-594bd6081af8" containerName="main" Jul 16 00:23:19.715892 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.715820 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="541e1318-864e-4e89-b75c-ff9e8632da09" containerName="main" Jul 16 00:23:19.720165 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.720145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.723173 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.723141 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"kube-root-ca.crt\"" Jul 16 00:23:19.723298 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.723174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"default-dockercfg-b4v4l\"" Jul 16 00:23:19.723298 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.723142 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"openshift-service-ca.crt\"" Jul 16 00:23:19.723417 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.723298 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26b6c6\"/\"sub-test-model-26b6c6-kserve-self-signed-certs\"" Jul 16 00:23:19.730656 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.730637 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:23:19.736684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.736663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:23:19.808393 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvhds\" (UniqueName: \"kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.808485 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.808485 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.808563 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.808563 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.808640 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.808574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909478 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909624 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvhds\" (UniqueName: \"kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909624 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909902 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909889 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.909988 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909966 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.910046 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.909991 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.911900 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.911882 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.912153 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.912137 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:19.918413 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:19.918392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvhds\" (UniqueName: \"kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds\") pod \"sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:20.030806 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:20.030781 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:20.158333 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:20.158310 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:23:20.160162 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:23:20.160136 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod840f9001_71b6_44f6_96a8_9443830a217f.slice/crio-78474427e2a24a5149fdf948eb1de545f91f9cc5ec9783faed6acad043101243 WatchSource:0}: Error finding container 78474427e2a24a5149fdf948eb1de545f91f9cc5ec9783faed6acad043101243: Status 404 returned error can't find the container with id 78474427e2a24a5149fdf948eb1de545f91f9cc5ec9783faed6acad043101243 Jul 16 00:23:20.932744 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:20.932711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" event={"ID":"840f9001-71b6-44f6-96a8-9443830a217f","Type":"ContainerStarted","Data":"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf"} Jul 16 00:23:20.933110 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:20.932760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" event={"ID":"840f9001-71b6-44f6-96a8-9443830a217f","Type":"ContainerStarted","Data":"78474427e2a24a5149fdf948eb1de545f91f9cc5ec9783faed6acad043101243"} Jul 16 00:23:20.950390 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:20.950338 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" podStartSLOduration=1.950322857 podStartE2EDuration="1.950322857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:23:20.949449212 +0000 UTC m=+2156.004779278" watchObservedRunningTime="2026-07-16 00:23:20.950322857 +0000 UTC m=+2156.005652979" Jul 16 00:23:30.031133 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:30.031095 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:30.031540 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:30.031144 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:30.043354 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:30.043328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:30.989122 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:30.989096 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:23:51.012369 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.012337 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:23:51.015646 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.015629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.018835 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.018809 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"kube-root-ca.crt\"" Jul 16 00:23:51.020505 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.020480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"default-dockercfg-2hxht\"" Jul 16 00:23:51.020631 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.020503 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"sub-test-model-c688a9-kserve-self-signed-certs\"" Jul 16 00:23:51.020681 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.020644 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"openshift-service-ca.crt\"" Jul 16 00:23:51.036792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036763 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.036792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.036931 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.036931 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.036931 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.037031 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.036937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r6md\" (UniqueName: \"kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.043679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.043657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:23:51.050706 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.050679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:23:51.138061 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r6md\" (UniqueName: \"kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.138194 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.138194 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.138194 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.138368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.138368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.139309 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138901 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.139309 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.138948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.139309 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.139004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.141353 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.141325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.141854 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.141829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.151554 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.151529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r6md\" (UniqueName: \"kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md\") pod \"sub-test-model-c688a9-kserve-6574df6d65-m448z\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.324975 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.324916 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:23:51.456682 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:51.456657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:23:51.460460 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:23:51.460430 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d8412fb_478c_4cfa_a47b_38d64c66ef94.slice/crio-2781a4369c635e4f550c176d36bd334f258ab897deb7465e1887f3bc59ae9972 WatchSource:0}: Error finding container 2781a4369c635e4f550c176d36bd334f258ab897deb7465e1887f3bc59ae9972: Status 404 returned error can't find the container with id 2781a4369c635e4f550c176d36bd334f258ab897deb7465e1887f3bc59ae9972 Jul 16 00:23:52.075891 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:52.075860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" event={"ID":"8d8412fb-478c-4cfa-a47b-38d64c66ef94","Type":"ContainerStarted","Data":"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd"} Jul 16 00:23:52.076287 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:52.075905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" event={"ID":"8d8412fb-478c-4cfa-a47b-38d64c66ef94","Type":"ContainerStarted","Data":"2781a4369c635e4f550c176d36bd334f258ab897deb7465e1887f3bc59ae9972"} Jul 16 00:23:52.094199 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:23:52.094136 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" podStartSLOduration=2.09412275 podStartE2EDuration="2.09412275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:23:52.092850371 +0000 UTC m=+2187.148180478" watchObservedRunningTime="2026-07-16 00:23:52.09412275 +0000 UTC m=+2187.149452822" Jul 16 00:24:01.325706 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:01.325677 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:24:01.325706 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:01.325710 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:24:01.337754 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:01.337729 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:24:02.133698 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:02.133673 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:24:44.993611 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:44.993556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:24:48.524465 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:48.524423 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:24:48.542898 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:48.542874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:24:49.500941 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:49.500902 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:24:49.525107 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:49.525077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:24:59.262177 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:59.262147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:24:59.362489 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:24:59.362461 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:00.322900 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:00.322871 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:03.544542 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.544492 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" podUID="840f9001-71b6-44f6-96a8-9443830a217f" containerName="main" containerID="cri-o://88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf" gracePeriod=45 Jul 16 00:25:03.790243 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.790209 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:25:03.930880 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.930813 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.930880 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.930849 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvhds\" (UniqueName: \"kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.931087 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.930892 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.931087 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.930914 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.931087 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.930955 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.931255 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931100 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:03.931255 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931155 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm\") pod \"840f9001-71b6-44f6-96a8-9443830a217f\" (UID: \"840f9001-71b6-44f6-96a8-9443830a217f\") " Jul 16 00:25:03.931255 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931205 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home" (OuterVolumeSpecName: "home") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:03.931255 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931216 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache" (OuterVolumeSpecName: "model-cache") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:03.931471 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931442 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:03.931471 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931459 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:03.931471 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.931468 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:03.932979 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.932954 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm" (OuterVolumeSpecName: "dshm") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:03.933091 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.933070 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds" (OuterVolumeSpecName: "kube-api-access-lvhds") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "kube-api-access-lvhds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:25:03.933133 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:03.933108 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "840f9001-71b6-44f6-96a8-9443830a217f" (UID: "840f9001-71b6-44f6-96a8-9443830a217f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:25:04.032589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.032564 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvhds\" (UniqueName: \"kubernetes.io/projected/840f9001-71b6-44f6-96a8-9443830a217f-kube-api-access-lvhds\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.032589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.032584 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/840f9001-71b6-44f6-96a8-9443830a217f-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.032589 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.032593 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/840f9001-71b6-44f6-96a8-9443830a217f-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.402785 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.402758 2580 generic.go:356] "Generic (PLEG): container finished" podID="840f9001-71b6-44f6-96a8-9443830a217f" containerID="88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf" exitCode=0 Jul 16 00:25:04.402910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.402802 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" Jul 16 00:25:04.402910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.402830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" event={"ID":"840f9001-71b6-44f6-96a8-9443830a217f","Type":"ContainerDied","Data":"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf"} Jul 16 00:25:04.402910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.402862 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl" event={"ID":"840f9001-71b6-44f6-96a8-9443830a217f","Type":"ContainerDied","Data":"78474427e2a24a5149fdf948eb1de545f91f9cc5ec9783faed6acad043101243"} Jul 16 00:25:04.402910 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.402878 2580 scope.go:117] "RemoveContainer" containerID="88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf" Jul 16 00:25:04.411697 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.411680 2580 scope.go:117] "RemoveContainer" containerID="88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf" Jul 16 00:25:04.411977 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:25:04.411955 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf\": container with ID starting with 88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf not found: ID does not exist" containerID="88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf" Jul 16 00:25:04.412037 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.411986 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf"} err="failed to get container status \"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf\": rpc error: code = NotFound desc = could not find container \"88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf\": container with ID starting with 88502fdec503482f443df617aaf683362b0ab7ce52ac0b2c2e62c91ca7ce9ddf not found: ID does not exist" Jul 16 00:25:04.424655 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.424632 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:25:04.429058 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.429040 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26b6c6/sub-test-model-26b6c6-kserve-5d55fdc8f7-rcwdl"] Jul 16 00:25:04.522684 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.522648 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" podUID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" containerName="main" containerID="cri-o://e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd" gracePeriod=45 Jul 16 00:25:04.770997 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.770974 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:25:04.840182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840150 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840298 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840197 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840298 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840252 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840298 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840281 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r6md\" (UniqueName: \"kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840535 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840311 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840535 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840342 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache\") pod \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\" (UID: \"8d8412fb-478c-4cfa-a47b-38d64c66ef94\") " Jul 16 00:25:04.840535 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840404 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home" (OuterVolumeSpecName: "home") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:04.840674 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840532 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:04.840674 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840612 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:04.840674 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840668 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.840774 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.840691 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.842368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.842341 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:25:04.842476 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.842391 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm" (OuterVolumeSpecName: "dshm") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:25:04.842476 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.842434 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md" (OuterVolumeSpecName: "kube-api-access-5r6md") pod "8d8412fb-478c-4cfa-a47b-38d64c66ef94" (UID: "8d8412fb-478c-4cfa-a47b-38d64c66ef94"). InnerVolumeSpecName "kube-api-access-5r6md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:25:04.941538 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.941463 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r6md\" (UniqueName: \"kubernetes.io/projected/8d8412fb-478c-4cfa-a47b-38d64c66ef94-kube-api-access-5r6md\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.941538 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.941487 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8412fb-478c-4cfa-a47b-38d64c66ef94-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.941538 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.941497 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:04.941538 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:04.941505 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8412fb-478c-4cfa-a47b-38d64c66ef94-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:25:05.409166 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.409134 2580 generic.go:356] "Generic (PLEG): container finished" podID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" containerID="e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd" exitCode=0 Jul 16 00:25:05.409365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.409173 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" Jul 16 00:25:05.409365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.409213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" event={"ID":"8d8412fb-478c-4cfa-a47b-38d64c66ef94","Type":"ContainerDied","Data":"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd"} Jul 16 00:25:05.409365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.409270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z" event={"ID":"8d8412fb-478c-4cfa-a47b-38d64c66ef94","Type":"ContainerDied","Data":"2781a4369c635e4f550c176d36bd334f258ab897deb7465e1887f3bc59ae9972"} Jul 16 00:25:05.409365 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.409292 2580 scope.go:117] "RemoveContainer" containerID="e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd" Jul 16 00:25:05.418030 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.418010 2580 scope.go:117] "RemoveContainer" containerID="e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd" Jul 16 00:25:05.418277 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:25:05.418257 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd\": container with ID starting with e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd not found: ID does not exist" containerID="e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd" Jul 16 00:25:05.418318 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.418286 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd"} err="failed to get container status \"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd\": rpc error: code = NotFound desc = could not find container \"e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd\": container with ID starting with e3fc88960cdc2573c61f287e16e4f44deb28653f6ee854925032ef63aba9f3dd not found: ID does not exist" Jul 16 00:25:05.430543 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.430245 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:25:05.436879 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.436859 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/sub-test-model-c688a9-kserve-6574df6d65-m448z"] Jul 16 00:25:05.507795 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.507773 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="840f9001-71b6-44f6-96a8-9443830a217f" path="/var/lib/kubelet/pods/840f9001-71b6-44f6-96a8-9443830a217f/volumes" Jul 16 00:25:05.508147 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:05.508125 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" path="/var/lib/kubelet/pods/8d8412fb-478c-4cfa-a47b-38d64c66ef94/volumes" Jul 16 00:25:07.140773 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:07.140740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:07.238726 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:07.238700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:28.300773 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.300730 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301055 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f9001-71b6-44f6-96a8-9443830a217f" containerName="main" Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301065 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f9001-71b6-44f6-96a8-9443830a217f" containerName="main" Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301087 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" containerName="main" Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301093 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" containerName="main" Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301179 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f9001-71b6-44f6-96a8-9443830a217f" containerName="main" Jul 16 00:25:28.301322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.301193 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8412fb-478c-4cfa-a47b-38d64c66ef94" containerName="main" Jul 16 00:25:28.305941 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.305919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.310182 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.310159 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"default-dockercfg-2hxht\"" Jul 16 00:25:28.310299 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.310188 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"openshift-service-ca.crt\"" Jul 16 00:25:28.310299 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.310237 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"kube-root-ca.crt\"" Jul 16 00:25:28.310299 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.310257 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c688a9\"/\"rate-test-model-c688a9-kserve-self-signed-certs\"" Jul 16 00:25:28.315863 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.315839 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:25:28.339014 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.338990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:28.431984 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.431953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.432123 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.432099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.432206 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.432155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qjr\" (UniqueName: \"kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.432297 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.432244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.432355 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.432316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.432355 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.432349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533621 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533621 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533818 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qjr\" (UniqueName: \"kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533818 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533818 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.533818 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.533733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.534048 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.534012 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.534087 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.534044 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.534125 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.534102 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.535890 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.535872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.536140 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.536122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.541735 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.541712 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qjr\" (UniqueName: \"kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr\") pod \"rate-test-model-c688a9-kserve-5b94b778cf-wwlwt\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.616687 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.616637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:28.743860 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:28.743837 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:25:28.746408 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:25:28.746379 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77f19601_0c08_4da7_bacd_29dc2ae27919.slice/crio-e8d44b453654d2689d346cf66125fd35b1d37b0f66c1ee77cc4e0c27d9b7fdd6 WatchSource:0}: Error finding container e8d44b453654d2689d346cf66125fd35b1d37b0f66c1ee77cc4e0c27d9b7fdd6: Status 404 returned error can't find the container with id e8d44b453654d2689d346cf66125fd35b1d37b0f66c1ee77cc4e0c27d9b7fdd6 Jul 16 00:25:29.516201 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:29.516168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" event={"ID":"77f19601-0c08-4da7-bacd-29dc2ae27919","Type":"ContainerStarted","Data":"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3"} Jul 16 00:25:29.516609 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:29.516207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" event={"ID":"77f19601-0c08-4da7-bacd-29dc2ae27919","Type":"ContainerStarted","Data":"e8d44b453654d2689d346cf66125fd35b1d37b0f66c1ee77cc4e0c27d9b7fdd6"} Jul 16 00:25:29.534600 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:29.534555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" podStartSLOduration=1.534542476 podStartE2EDuration="1.534542476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:25:29.533627191 +0000 UTC m=+2284.588957263" watchObservedRunningTime="2026-07-16 00:25:29.534542476 +0000 UTC m=+2284.589872549" Jul 16 00:25:38.616792 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:38.616759 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:38.617295 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:38.616802 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:38.629234 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:38.629204 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:39.568993 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:39.568967 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:25:44.283913 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:44.283883 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:25:44.384401 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:25:44.384375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:10.709713 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:10.709686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:25.491807 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:25.491703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:26.503470 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:26.503439 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:26:26.531318 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:26.531276 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:41.522936 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.522875 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" podUID="77f19601-0c08-4da7-bacd-29dc2ae27919" containerName="main" containerID="cri-o://bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3" gracePeriod=45 Jul 16 00:26:41.761635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.761613 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:26:41.831179 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831116 2580 generic.go:356] "Generic (PLEG): container finished" podID="77f19601-0c08-4da7-bacd-29dc2ae27919" containerID="bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3" exitCode=0 Jul 16 00:26:41.831179 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831154 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" Jul 16 00:26:41.831350 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" event={"ID":"77f19601-0c08-4da7-bacd-29dc2ae27919","Type":"ContainerDied","Data":"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3"} Jul 16 00:26:41.831350 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt" event={"ID":"77f19601-0c08-4da7-bacd-29dc2ae27919","Type":"ContainerDied","Data":"e8d44b453654d2689d346cf66125fd35b1d37b0f66c1ee77cc4e0c27d9b7fdd6"} Jul 16 00:26:41.831350 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831286 2580 scope.go:117] "RemoveContainer" containerID="bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3" Jul 16 00:26:41.831611 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831626 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9qjr\" (UniqueName: \"kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831698 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831743 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831967 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831776 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm\") pod \"77f19601-0c08-4da7-bacd-29dc2ae27919\" (UID: \"77f19601-0c08-4da7-bacd-29dc2ae27919\") " Jul 16 00:26:41.831967 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831802 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache" (OuterVolumeSpecName: "model-cache") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:41.831967 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831886 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home" (OuterVolumeSpecName: "home") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:41.832142 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.831997 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:41.832368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.832140 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-model-cache\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:41.832368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.832159 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-home\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:41.832368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.832201 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-tmp-dir\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:41.833889 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.833861 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm" (OuterVolumeSpecName: "dshm") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:41.833889 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.833868 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:26:41.834022 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.834002 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr" (OuterVolumeSpecName: "kube-api-access-s9qjr") pod "77f19601-0c08-4da7-bacd-29dc2ae27919" (UID: "77f19601-0c08-4da7-bacd-29dc2ae27919"). InnerVolumeSpecName "kube-api-access-s9qjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:26:41.851117 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.851099 2580 scope.go:117] "RemoveContainer" containerID="bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3" Jul 16 00:26:41.851397 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:26:41.851378 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3\": container with ID starting with bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3 not found: ID does not exist" containerID="bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3" Jul 16 00:26:41.851444 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.851405 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3"} err="failed to get container status \"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3\": rpc error: code = NotFound desc = could not find container \"bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3\": container with ID starting with bedc8d3aaabf9f75f7786a505761ae10d4fa0c5da7a36eba6db23f41351382a3 not found: ID does not exist" Jul 16 00:26:41.933354 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.933331 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9qjr\" (UniqueName: \"kubernetes.io/projected/77f19601-0c08-4da7-bacd-29dc2ae27919-kube-api-access-s9qjr\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:41.933354 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.933352 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f19601-0c08-4da7-bacd-29dc2ae27919-tls-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:41.933477 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:41.933361 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f19601-0c08-4da7-bacd-29dc2ae27919-dshm\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:42.156178 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.156153 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:26:42.163574 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.163545 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c688a9/rate-test-model-c688a9-kserve-5b94b778cf-wwlwt"] Jul 16 00:26:42.830633 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.830609 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7251-139645388456272"] Jul 16 00:26:42.830976 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.830918 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f19601-0c08-4da7-bacd-29dc2ae27919" containerName="main" Jul 16 00:26:42.830976 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.830931 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f19601-0c08-4da7-bacd-29dc2ae27919" containerName="main" Jul 16 00:26:42.831051 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.831014 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f19601-0c08-4da7-bacd-29dc2ae27919" containerName="main" Jul 16 00:26:42.835205 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.835176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:42.838686 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.838660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-brw85\"" Jul 16 00:26:42.850542 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.850522 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7251-139645388456272"] Jul 16 00:26:42.941535 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:42.941512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv842\" (UniqueName: \"kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842\") pod \"test-curl-7251-139645388456272\" (UID: \"04d37c89-b919-4b2e-92ee-72aa2e7d9da5\") " pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:43.042089 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.042051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv842\" (UniqueName: \"kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842\") pod \"test-curl-7251-139645388456272\" (UID: \"04d37c89-b919-4b2e-92ee-72aa2e7d9da5\") " pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:43.051927 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.051906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv842\" (UniqueName: \"kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842\") pod \"test-curl-7251-139645388456272\" (UID: \"04d37c89-b919-4b2e-92ee-72aa2e7d9da5\") " pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:43.144129 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.144068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:43.269662 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.269639 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7251-139645388456272"] Jul 16 00:26:43.271697 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:26:43.271670 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04d37c89_b919_4b2e_92ee_72aa2e7d9da5.slice/crio-529c4b3d19e8202913cf7ecca24fe9ad9431d443ee375a1c86e0e1d28a47e3eb WatchSource:0}: Error finding container 529c4b3d19e8202913cf7ecca24fe9ad9431d443ee375a1c86e0e1d28a47e3eb: Status 404 returned error can't find the container with id 529c4b3d19e8202913cf7ecca24fe9ad9431d443ee375a1c86e0e1d28a47e3eb Jul 16 00:26:43.273003 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.272982 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:26:43.508635 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.508610 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f19601-0c08-4da7-bacd-29dc2ae27919" path="/var/lib/kubelet/pods/77f19601-0c08-4da7-bacd-29dc2ae27919/volumes" Jul 16 00:26:43.844006 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:43.843926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7251-139645388456272" event={"ID":"04d37c89-b919-4b2e-92ee-72aa2e7d9da5","Type":"ContainerStarted","Data":"529c4b3d19e8202913cf7ecca24fe9ad9431d443ee375a1c86e0e1d28a47e3eb"} Jul 16 00:26:44.849739 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:44.849704 2580 generic.go:356] "Generic (PLEG): container finished" podID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" containerID="1c56ec1834085fd586a9968c3e0701257b549d97602fbe9b37c164d749783e1a" exitCode=0 Jul 16 00:26:44.850124 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:44.849750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7251-139645388456272" event={"ID":"04d37c89-b919-4b2e-92ee-72aa2e7d9da5","Type":"ContainerDied","Data":"1c56ec1834085fd586a9968c3e0701257b549d97602fbe9b37c164d749783e1a"} Jul 16 00:26:45.982046 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:45.982021 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:46.025842 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.025819 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7251-139645388456272_04d37c89-b919-4b2e-92ee-72aa2e7d9da5/test-curl-7251-139645388456272/0.log" Jul 16 00:26:46.063128 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.063101 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7251-139645388456272"] Jul 16 00:26:46.065752 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.065733 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv842\" (UniqueName: \"kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842\") pod \"04d37c89-b919-4b2e-92ee-72aa2e7d9da5\" (UID: \"04d37c89-b919-4b2e-92ee-72aa2e7d9da5\") " Jul 16 00:26:46.067933 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.067902 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7251-139645388456272"] Jul 16 00:26:46.068055 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.068025 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842" (OuterVolumeSpecName: "kube-api-access-rv842") pod "04d37c89-b919-4b2e-92ee-72aa2e7d9da5" (UID: "04d37c89-b919-4b2e-92ee-72aa2e7d9da5"). InnerVolumeSpecName "kube-api-access-rv842". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:26:46.166838 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.166785 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv842\" (UniqueName: \"kubernetes.io/projected/04d37c89-b919-4b2e-92ee-72aa2e7d9da5-kube-api-access-rv842\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:46.696603 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.696574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:46.859939 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.859917 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:46.860066 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:46.859938 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="529c4b3d19e8202913cf7ecca24fe9ad9431d443ee375a1c86e0e1d28a47e3eb" Jul 16 00:26:46.871340 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:26:46.871314 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7251-139645388456272\" is forbidden: User \"system:node:ip-10-0-129-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-211.ec2.internal' and this object" podUID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" pod="opendatahub/test-curl-7251-139645388456272" Jul 16 00:26:47.508343 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:47.508313 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" path="/var/lib/kubelet/pods/04d37c89-b919-4b2e-92ee-72aa2e7d9da5/volumes" Jul 16 00:26:54.284264 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.284211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:54.382368 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.382338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:54.615449 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.615373 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:26:54.615619 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.615595 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" containerID="cri-o://876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728" gracePeriod=30 Jul 16 00:26:54.628843 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.628817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:54.872164 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:54.872086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:55.403088 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:55.403055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:26:59.866718 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.866689 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:26:59.924536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.924455 2580 generic.go:356] "Generic (PLEG): container finished" podID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerID="876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728" exitCode=0 Jul 16 00:26:59.924536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.924505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerDied","Data":"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728"} Jul 16 00:26:59.924536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.924510 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" Jul 16 00:26:59.924790 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.924577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm" event={"ID":"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c","Type":"ContainerDied","Data":"7db462c2e35f2673413bf96f5839856756a576f40499d5a4e31cee115ef1d439"} Jul 16 00:26:59.924790 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.924600 2580 scope.go:117] "RemoveContainer" containerID="876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728" Jul 16 00:26:59.933172 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.933155 2580 scope.go:117] "RemoveContainer" containerID="03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63" Jul 16 00:26:59.940812 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.940797 2580 scope.go:117] "RemoveContainer" containerID="876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728" Jul 16 00:26:59.941064 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:26:59.941045 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728\": container with ID starting with 876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728 not found: ID does not exist" containerID="876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728" Jul 16 00:26:59.941135 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.941098 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728"} err="failed to get container status \"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728\": rpc error: code = NotFound desc = could not find container \"876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728\": container with ID starting with 876b40a1fd367866c405a17e7a786355517a3df15f7d50aaa73a4d02161df728 not found: ID does not exist" Jul 16 00:26:59.941135 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.941121 2580 scope.go:117] "RemoveContainer" containerID="03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63" Jul 16 00:26:59.941391 ip-10-0-129-211 kubenswrapper[2580]: E0716 00:26:59.941372 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63\": container with ID starting with 03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63 not found: ID does not exist" containerID="03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63" Jul 16 00:26:59.941447 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.941399 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63"} err="failed to get container status \"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63\": rpc error: code = NotFound desc = could not find container \"03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63\": container with ID starting with 03fb079ef9f66d04a3b4b79fd3f7cbd0f6915464ac141f9e327cc4a0280a1f63 not found: ID does not exist" Jul 16 00:26:59.977739 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977720 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.977836 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977760 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.977836 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977799 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.977836 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977828 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977856 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977890 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpsrz\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977917 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977949 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.977993 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo\") pod \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\" (UID: \"a2a4e1b8-cf14-4dc8-9506-dcd6835c986c\") " Jul 16 00:26:59.978252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978143 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:59.978252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978239 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:26:59.978363 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978331 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istiod-ca-cert\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:59.978363 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978348 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:26:59.978495 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978473 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:59.978560 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978502 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:59.978622 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.978597 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:59.979969 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.979950 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:26:59.980193 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.980178 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:26:59.980454 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.980436 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:26:59.980523 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:26:59.980445 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz" (OuterVolumeSpecName: "kube-api-access-mpsrz") pod "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" (UID: "a2a4e1b8-cf14-4dc8-9506-dcd6835c986c"). InnerVolumeSpecName "kube-api-access-mpsrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:27:00.079110 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079086 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-podinfo\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079110 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079107 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-data\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079117 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-workload-certs\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079125 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-credential-socket\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079133 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpsrz\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-kube-api-access-mpsrz\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079141 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-envoy\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.079252 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.079149 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c-istio-token\") on node \"ip-10-0-129-211.ec2.internal\" DevicePath \"\"" Jul 16 00:27:00.252468 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.252441 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:27:00.259699 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:00.259675 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-26b6c6-openshift-default-56f469b5c5-r4gqm"] Jul 16 00:27:01.509242 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:01.509180 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" path="/var/lib/kubelet/pods/a2a4e1b8-cf14-4dc8-9506-dcd6835c986c/volumes" Jul 16 00:27:02.832145 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:02.832113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:02.938447 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:02.938416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:03.146321 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:03.146248 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:03.411985 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:03.411905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:10.028857 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:10.028823 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t4gjr_0433cea8-009b-4694-9f3b-68da5b267357/server/0.log" Jul 16 00:27:10.465007 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:10.464930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4446c48-d59pk_f25211ed-8117-40e1-a947-012c37f2b045/copy-manifests/0.log" Jul 16 00:27:10.478628 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:10.478605 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4446c48-d59pk_f25211ed-8117-40e1-a947-012c37f2b045/manager/0.log" Jul 16 00:27:15.838193 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:15.838163 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ptrzx_e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f/istio-proxy/0.log" Jul 16 00:27:15.965451 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:15.965426 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44qlx_4509e29d-4571-423e-9129-1d1fb97a614a/discovery/0.log" Jul 16 00:27:16.207951 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:16.207890 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797ddd4965-75pss_4131bbca-92b1-4bc4-b3d4-323f0332c887/kube-auth-proxy/0.log" Jul 16 00:27:23.626422 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:23.626390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:23.746755 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:23.746725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:26.811092 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:26.811066 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xxp97_6ab8a1c8-57e8-47b4-999b-63f569069d7a/global-pull-secret-syncer/0.log" Jul 16 00:27:26.877617 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:26.877589 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" Jul 16 00:27:26.984741 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:26.984718 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-211.ec2.internal_5cd24277ff6091c18d8b9f1eee6e8097/haproxy/0.log" Jul 16 00:27:34.869259 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:34.869233 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/alertmanager/0.log" Jul 16 00:27:34.906499 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:34.906475 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/config-reloader/0.log" Jul 16 00:27:34.936740 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:34.936717 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/kube-rbac-proxy-web/0.log" Jul 16 00:27:34.961536 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:34.961518 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/kube-rbac-proxy/0.log" Jul 16 00:27:34.989827 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:34.989811 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/kube-rbac-proxy-metric/0.log" Jul 16 00:27:35.018331 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.018311 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/prom-label-proxy/0.log" Jul 16 00:27:35.044413 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.044398 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1c657173-40bb-488d-acce-8a6097d30328/init-config-reloader/0.log" Jul 16 00:27:35.116717 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.116700 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-56kxs_98b956d6-7d07-4587-a659-6fba4587f78f/kube-state-metrics/0.log" Jul 16 00:27:35.139185 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.139137 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-56kxs_98b956d6-7d07-4587-a659-6fba4587f78f/kube-rbac-proxy-main/0.log" Jul 16 00:27:35.164967 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.164949 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-56kxs_98b956d6-7d07-4587-a659-6fba4587f78f/kube-rbac-proxy-self/0.log" Jul 16 00:27:35.192335 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.192318 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fdcb84974-nknxh_dcf0282f-e149-4a75-afb9-2a5ac96371aa/metrics-server/0.log" Jul 16 00:27:35.256458 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.256439 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdjm5_d63be52a-76c0-4254-a95a-1cd47fe6692c/node-exporter/0.log" Jul 16 00:27:35.282721 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.282703 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdjm5_d63be52a-76c0-4254-a95a-1cd47fe6692c/kube-rbac-proxy/0.log" Jul 16 00:27:35.306469 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.306450 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdjm5_d63be52a-76c0-4254-a95a-1cd47fe6692c/init-textfile/0.log" Jul 16 00:27:35.530777 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.530756 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bxxhz_1ee6eb64-d6c4-45a6-a835-35829f78882b/kube-rbac-proxy-main/0.log" Jul 16 00:27:35.559644 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.559623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bxxhz_1ee6eb64-d6c4-45a6-a835-35829f78882b/kube-rbac-proxy-self/0.log" Jul 16 00:27:35.590732 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.590713 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bxxhz_1ee6eb64-d6c4-45a6-a835-35829f78882b/openshift-state-metrics/0.log" Jul 16 00:27:35.636993 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.636976 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/prometheus/0.log" Jul 16 00:27:35.678394 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.678375 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/config-reloader/0.log" Jul 16 00:27:35.709143 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.709122 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/thanos-sidecar/0.log" Jul 16 00:27:35.733845 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.733827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/kube-rbac-proxy-web/0.log" Jul 16 00:27:35.760448 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.760432 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/kube-rbac-proxy/0.log" Jul 16 00:27:35.786113 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.786071 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/kube-rbac-proxy-thanos/0.log" Jul 16 00:27:35.813873 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.813853 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a91adafa-f20a-4bbf-8a7d-c6a110bddfb0/init-config-reloader/0.log" Jul 16 00:27:35.964679 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.964661 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5887f979f7-7jc9c_73859a5b-1b20-435d-9683-49b92814504d/telemeter-client/0.log" Jul 16 00:27:35.990146 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:35.990130 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5887f979f7-7jc9c_73859a5b-1b20-435d-9683-49b92814504d/reload/0.log" Jul 16 00:27:36.013568 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.013548 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5887f979f7-7jc9c_73859a5b-1b20-435d-9683-49b92814504d/kube-rbac-proxy/0.log" Jul 16 00:27:36.698779 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.698753 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k"] Jul 16 00:27:36.699072 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699060 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" Jul 16 00:27:36.699117 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699074 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" Jul 16 00:27:36.699117 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699086 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" containerName="test-curl-7251-139645388456272" Jul 16 00:27:36.699117 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699092 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" containerName="test-curl-7251-139645388456272" Jul 16 00:27:36.699215 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699154 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a4e1b8-cf14-4dc8-9506-dcd6835c986c" containerName="istio-proxy" Jul 16 00:27:36.699215 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.699165 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d37c89-b919-4b2e-92ee-72aa2e7d9da5" containerName="test-curl-7251-139645388456272" Jul 16 00:27:36.701389 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.701374 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.704349 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.704333 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9plpw\"/\"kube-root-ca.crt\"" Jul 16 00:27:36.704499 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.704482 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9plpw\"/\"default-dockercfg-2clp9\"" Jul 16 00:27:36.705656 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.705636 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9plpw\"/\"openshift-service-ca.crt\"" Jul 16 00:27:36.714006 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.713986 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k"] Jul 16 00:27:36.766124 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.766097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-proc\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.766238 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.766136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-lib-modules\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.766285 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.766249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-podres\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.766322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.766283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bjc2\" (UniqueName: \"kubernetes.io/projected/6c288556-61f9-4123-9175-f4bee2378cca-kube-api-access-6bjc2\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.766322 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.766313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-sys\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867564 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-podres\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867669 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bjc2\" (UniqueName: \"kubernetes.io/projected/6c288556-61f9-4123-9175-f4bee2378cca-kube-api-access-6bjc2\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867669 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-sys\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867669 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-proc\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-lib-modules\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867722 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-podres\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867744 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-sys\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867772 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-proc\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.867814 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.867799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c288556-61f9-4123-9175-f4bee2378cca-lib-modules\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:36.876942 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:36.876914 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bjc2\" (UniqueName: \"kubernetes.io/projected/6c288556-61f9-4123-9175-f4bee2378cca-kube-api-access-6bjc2\") pod \"perf-node-gather-daemonset-2fd2k\" (UID: \"6c288556-61f9-4123-9175-f4bee2378cca\") " pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:37.011854 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:37.011832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:37.140072 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:37.140049 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k"] Jul 16 00:27:37.142038 ip-10-0-129-211 kubenswrapper[2580]: W0716 00:27:37.142013 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c288556_61f9_4123_9175_f4bee2378cca.slice/crio-ec8ddfa92db1ab227d841095660f227a4163c608dda93b39961a4d0dc2d41a88 WatchSource:0}: Error finding container ec8ddfa92db1ab227d841095660f227a4163c608dda93b39961a4d0dc2d41a88: Status 404 returned error can't find the container with id ec8ddfa92db1ab227d841095660f227a4163c608dda93b39961a4d0dc2d41a88 Jul 16 00:27:38.104001 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:38.103967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" event={"ID":"6c288556-61f9-4123-9175-f4bee2378cca","Type":"ContainerStarted","Data":"415192a69751c06f713d8ac91cf5a8214f0e53267a4a4972c6130c9eaf418d4e"} Jul 16 00:27:38.104472 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:38.104064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" event={"ID":"6c288556-61f9-4123-9175-f4bee2378cca","Type":"ContainerStarted","Data":"ec8ddfa92db1ab227d841095660f227a4163c608dda93b39961a4d0dc2d41a88"} Jul 16 00:27:38.104472 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:38.104086 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:38.125323 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:38.125278 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" podStartSLOduration=2.125263002 podStartE2EDuration="2.125263002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:27:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:27:38.124588072 +0000 UTC m=+2413.179918144" watchObservedRunningTime="2026-07-16 00:27:38.125263002 +0000 UTC m=+2413.180593075" Jul 16 00:27:40.244723 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:40.244700 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4pkn_088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1/dns/0.log" Jul 16 00:27:40.278396 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:40.278374 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4pkn_088d1414-8bf4-4f20-bb7f-f1b2dc7e72b1/kube-rbac-proxy/0.log" Jul 16 00:27:40.413233 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:40.413194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b96kf_77d8b75b-5d7b-4ec0-8b92-c48b529da598/dns-node-resolver/0.log" Jul 16 00:27:41.122188 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:41.122164 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2v9r6_b7fcbfe1-67e5-4f4f-bfec-e8577a815dd4/node-ca/0.log" Jul 16 00:27:42.245819 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:42.245793 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ptrzx_e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f/istio-proxy/0.log" Jul 16 00:27:42.341029 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:42.341009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44qlx_4509e29d-4571-423e-9129-1d1fb97a614a/discovery/0.log" Jul 16 00:27:42.395656 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:42.395638 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797ddd4965-75pss_4131bbca-92b1-4bc4-b3d4-323f0332c887/kube-auth-proxy/0.log" Jul 16 00:27:43.337311 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:43.337287 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ps92_eca580ce-12b8-42b5-b4b6-1cba272266ed/serve-healthcheck-canary/0.log" Jul 16 00:27:44.000012 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:43.999981 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcvxr_eef93151-d711-40ad-bb7a-8c6a4e52f08d/kube-rbac-proxy/0.log" Jul 16 00:27:44.031956 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:44.031931 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcvxr_eef93151-d711-40ad-bb7a-8c6a4e52f08d/exporter/0.log" Jul 16 00:27:44.066259 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:44.066236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcvxr_eef93151-d711-40ad-bb7a-8c6a4e52f08d/extractor/0.log" Jul 16 00:27:44.117965 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:44.117948 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9plpw/perf-node-gather-daemonset-2fd2k" Jul 16 00:27:44.452201 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:44.452138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l2mk_7eebb966-f5ca-42e6-8add-3fc2f7d25a2e/konnectivity-agent/0.log" limit=1024 Jul 16 00:27:47.105640 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.105539 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ptrzx_e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f/istio-proxy/0.log" Jul 16 00:27:47.133695 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.133662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t4gjr_0433cea8-009b-4694-9f3b-68da5b267357/server/0.log" Jul 16 00:27:47.224614 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.224593 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44qlx_4509e29d-4571-423e-9129-1d1fb97a614a/discovery/0.log" Jul 16 00:27:47.290885 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.290861 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797ddd4965-75pss_4131bbca-92b1-4bc4-b3d4-323f0332c887/kube-auth-proxy/0.log" Jul 16 00:27:47.624177 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.624157 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4446c48-d59pk_f25211ed-8117-40e1-a947-012c37f2b045/manager/0.log" Jul 16 00:27:47.661099 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:47.661079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4446c48-d59pk_f25211ed-8117-40e1-a947-012c37f2b045/copy-manifests/0.log" Jul 16 00:27:48.875532 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:48.875501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ptrzx_e58a1ce4-d2d3-4e7e-8b69-f5d3acdf0c4f/istio-proxy/0.log" Jul 16 00:27:48.976021 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:48.975996 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44qlx_4509e29d-4571-423e-9129-1d1fb97a614a/discovery/0.log" Jul 16 00:27:49.029442 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:49.029420 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797ddd4965-75pss_4131bbca-92b1-4bc4-b3d4-323f0332c887/kube-auth-proxy/0.log" Jul 16 00:27:49.044315 ip-10-0-129-211 kubenswrapper[2580]: I0716 00:27:49.044297 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-755cb7796d-7btms_18134957-5a78-40aa-9f86-6a37cebe12c5/manager/0.log"